Your blog deserves a great Content Delivery Network

The Thinker by Rodin

While I do a lot of blogging, I suck at marketing my blog. Oh, I do look at who’s viewing my blog and check my statistics daily, and often more than once a day. Google Analytics provides a wealth of data on my web hits, and StatCounter is useful to see what was recently read. Aside from dressing up my blog’s sidebars with marketing stuff and making sure my content is easily accessible as a newsfeed, I can’t seem to be bothered to do much else.

Part of the problem is that my blog serves principally to keep me amused and to stave off boredom. If readers find an occasional post worthy of a Facebook Like or a Share, that’s nice, but I don’t lose sleep when they don’t. You would think that as a software engineer and someone who spent ten years directing the management of the largest web site in the U.S. Department of the Interior, I might find this web marketing business pretty easy. But one thing I learned early on is if you have great content, the marketing kind of takes care of itself.

In that job I simply worked to make the content more readily accessible and to make sure that the data was easily consumed. I spent much of my ten years there leading an effort to make the site’s data accessible as a set of web services. In this sense I do know marketing. When I left these new web services constituted the third most accessed site for my agency, in spite of not having existed just a few years earlier.

On this blog though my traffic is pretty anemic, particularly during the summer. There are things I could do to get more hits: shorter posts, more topical posts, turn it into more of a stream of consciousness blog and link ruthlessly to posts in other blogs, which seems to be the way blog aggregators like Tumblr work. Doing this though would ruin blogging for me. It might be successful, but I wouldn’t care. I’d be bored with my own blog.

During one of the recent Net Neutrality debates I mentioned that the Internet was already not net neutral. If you can afford little, you may (shudder) use an Earthlink dial-up account and watch web pages slowly draw themselves like they did in 1995. If you can afford $100 a month or more for Internet, or live in a place like Kansas City where you can get Google Fiber, you can cruise the Internet at 100MB per second or more. Some people have 1GB/sec connections.

If you have your own web site you also have some factors that limit the speed of your website. That’s the case with this blog. I host the site on hostgator.com, which is a really good shared web host. What’s not optimal about Hostgator is that while it can reliably serve most content at $5 or so a month, getting the data between its servers and your computer can be like going through every traffic light in town to get home from work as opposed to taking the expressway. It typically took eight or more “hops” to get my blog posts to my computer. A “hop” in this case means a router, which is effectively a traffic light as it routes parts of web pages from one place to another. According to Google Analytics that it took about ten seconds to load one of my web pages. Most of that was due to all those routers that had to be traversed.

So it finally dawned on me that this was probably a significant reason my traffic is declining. Google is looking at the hassle factor at getting content from my site, and is probably lowering my search rankings because of it. Aware of the problem for several years I have used CloudFlare to try to speed up the serving of my content. CloudFlare is a content delivery network or CDN. It specializes in reducing the number of traffic lights and making sure that my content goes through crazily fast connections, usually one physically close to where you are. Hostgator (and a lot of web hosts) offer CloudFlare for free to its customers. CloudFlare like every CDN sells a more expansive service for those with deeper pockets.

I had outsourced my CDN to CloudFlare, but I never really went back to look to see if it was doing a good job. There are probably things I could do to cache more of my content on CloudFlare’s servers (probably for money) but mostly I stuck with its defaults and ignored it. However, when I looked at Google Analytics, my average page load time was still stuck at around ten seconds.

Ten seconds is a long time to wait for content these days. So I figured I was probably losing a lot of readers because they lose patience and go elsewhere, particularly mobile users. We want every web page to load like a Google web page: fully dress itself for our eyes in a couple of seconds or less.

But not my blog. It was like a horse-drawn milk wagon compared with a racing car. Actually, this describes a lot of sites on the web, particularly Mom and Pop affairs where the owners know little or nothing about web architecture.

I decided to put on my software engineering hat, and started researching CDNs some more. There’s a lot of competition in the market, mostly aimed at well moneyed corporations. I’m just a little blog, however. And this blog runs on WordPress. What options do I have for a swift CDN that won’t cost me an arm and a leg? CloudFlare was free but it clearly wasn’t doing the job.

After some research I settled on MaxCND.com. For about $9 a month it will serve my pages quick. Of course if traffic increases a whole lot it could get a lot more expensive. But if I am content to use principally their servers in Europe and the USA (which is most of my readers) and I expect a terabyte or less of bandwidth a month then $9 a month should be fine. I can afford that. My pages seem to load in about 3 seconds now. A lot of the sidebar stuff comes from elsewhere, so that slows things down a bit. But the main content, if it is cached, takes about a second to load. That’s pretty impressive for $9 a month. And this fast speed might draw in new readers.

So far it’s looking good. Today’s traffic is roughly double what it was two days ago. Over time Google may take notice and rank my posts higher in their search engine. Here’s hoping.

Does your blog or website need a CDN too? It can’t hurt if you can afford it, and it can’t hurt to do your research and see which CDN is best optimized for your kind of content. MaxCDN has a plug in that works with WordPress to facilitate sharing. It was a little tedious to get it configured but the instructions were clear enough. Some of it is kind of wonky (how many people know what minifying is anyhow?) but the more technical you are the more you can fine tune things.

Please note you don’t need a CDN if you are using a blogging platform like Tumblr, BlogSpot or WordPress.com. They are already effectively CDN platforms as well as blogging sites. But if you host your own site and you want to increase traffic, integrating your site with the right CDN may be the most cost effective way to go.

I’ll be watching my metrics and perhaps reporting success or failure in the months ahead. So far the signs look good.

The Internet is already not net neutral

The Thinker by Rodin

Upset by proposals by the Federal Communications Commission to create “express lanes” on the Internet? If the current proposal now out for public comment becomes a rule, it would allow Internet Service Providers (ISPs) like Verizon and Comcast to charge a fee to those web sites that want faster content delivery.

This is the opposite of net neutrality, which is the principle that all web content should be delivered by an ISP at the same speed. (Actually, it’s at the same bandwidth, since all network traffic is effectively at the speed of light.) The argument goes that without net neutrality, those companies with deeper pockets, particularly those who are already established, such as Netflix, have an unfair competitive advantage over other services or start ups without such deep pockets. It’s a concern I certainly share, so much so that I first blogged about it in 2006. Bottom line: I am still concerned and I think this proposal must be fought.

What I didn’t write about back in 2006 was that there was no net neutrality back then either. Effectively, bandwidth is already discriminatory because it is based on ability to pay. It’s just based on your ability to pay, not the content provider’s. For example, Verizon has basically four tiers of Internet service from it’s “high speed” service (actually it’s lowest speed service) where content delivery does not exceed 1MB per second to its “high speed Internet enhanced” service where you can download at up to 15MB per second. It’s hard to quantify what the cost of the 1MB/sec plan is compared to the 15MB/sec plan, because it depends on many factors including what bundle you may or may not choose. Suffice to say if you want a 15MB/sec service, you will pay more than a 1MB/sec service. So if streaming Netflix is critical to you, consider their 15MB/sec service. (Of course, this assumes that the port between Verizon and Netflix can handle 15MB/sec. If it can’t then there is no point in paying Verizon the premium.)

You can think of the Internet connection from your ISP like a water pipe. If the water pipe is big (and the water pressure is high enough) you can get more water per second through a bigger pipe. What the FCC is proposing is to take this pipe and put two pipes inside it. One is a fat pipe that will serve certain content very quickly, the “fast lane”. The other smaller pipe is for those who can’t afford to pay ISPs these premiums, i.e. the “slow lane”. Since I live in traffic-congested Washington D.C., I think of the “fast lane” as the pricey HOT (High Occupancy Toll) lanes on the beltway, and the “slow lane” as the toll free and usually congested other lanes. It’s not hard to imagine the Internet feeling a lot like it did in 1995, when the hourglass was principally what you saw in your web browser. Pages took forever to load, if they ever did. For those of us who remember those days, revisiting them sounds quite frightful. ISPs would have every incentive to throttle the slow lanes, because it would mean that web content providers would come to them and negotiate to use their fast lanes. In addition, they would have little incentive to increase bandwidth for their customers overall, but plenty of profit to funnel back to stockholders from those that pay for fast lanes. It is the antithesis of what the Internet is about.

So already there is no net neutrality of content delivery, unless you have an ISP that provides a “one speed for all customers” plan. The issue is not content delivery; it is the speed of particular content distribution within the ISP’s network. Which brings up another less noticed way that the Internet is not equal. It has to do with Content Delivery Networks (CDNs).

If you access my blog with a browser you will notice it takes a while to render a web page. Why is that? It’s because I don’t pay for a content delivery network. I did a test from home on accessing my web site. I had to go through 13 routers (switches on the internet) between my home computer and my web host:

1 wireless_broadband_router (192.168.1.1) 0.525 ms 0.244 ms 0.216 ms
2 l100.washdc-vfttp-93.verizon-gni.net (173.66.179.1) 7.083 ms 7.095 ms 8.161 ms
3 g1-5-3-0.washdc-lcr-21.verizon-gni.net (130.81.216.76) 9.435 ms 12.101 ms 12.305 ms
4 ae7-0.res-bb-rtr1.verizon-gni.net (130.81.174.208) 9.731 ms
   so-12-1-0-0.res-bb-rtr1.verizon-gni.net (130.81.151.230) 27.151 ms
   ae7-0.res-bb-rtr1.verizon-gni.net (130.81.174.208) 8.855 ms
5 0.ae1.xl1.iad8.alter.net (140.222.226.149) 10.166 ms
   0.ae5.xl1.iad8.alter.net (152.63.8.121) 9.396 ms 10.254 ms
6 0.xe-8-3-1.gw12.iad8.iad8.alter.net (152.63.37.14) 9.610 ms
   0.xe-8-0-0.gw12.iad8.alter.net (152.63.35.134) 9.693 ms
   0.xe-10-1-0.gw12.iad8.alter.net (152.63.35.102) 10.872 ms
7 customer.alter.net.customer.alter.net (152.179.50.206) 8.733 ms 10.023 ms 9.717 ms
8 he-2-4-0-0-cr01.ashburn.va.ibone.comcast.net (68.86.83.65) 10.252 ms
   he-2-6-0-0-cr01.ashburn.va.ibone.comcast.net (68.86.83.73) 14.819 ms
   he-2-5-0-0-cr01.ashburn.va.ibone.comcast.net (68.86.83.69) 12.388 ms
9 he-4-3-0-0-cr01.56marietta.ga.ibone.comcast.net (68.86.89.150) 39.468 ms 42.618 ms 37.101 ms
10 he-1-12-0-0-cr01.dallas.tx.ibone.comcast.net (68.86.88.234) 42.852 ms 45.176 ms 44.283 ms
11 be-22-pe01.houston.tx.ibone.comcast.net (68.86.85.174) 50.270 ms 49.438 ms 50.270 ms
12 as8075-1.2001sixthave.wa.ibone.comcast.net (75.149.230.54) 49.692 ms 85.009 ms 50.379 ms
13 216.117.50.142 (216.117.50.142) 49.597 ms

Electrons still travel at the speed of light, but they are thirteen stoplights between my computer and my web server, at least for me. You can see how long my request took at each stop. For example, hop 13 took 49.597 milliseconds. Add up all the milliseconds to see how long it took for me to get to my site. If you do the same thing, the number of hops will probably vary, along with the access time. In short, it’s relatively slow to get to, which alone may explain why my traffic is down. People are impatient when they click on a link to my site from a search index. So they go elsewhere or get an effective CDN by using a subscription service to read content like Feedburner or feedly.com.

This is not much of a problem if I go to google.com. Here is the route:

1 wireless_broadband_router (192.168.1.1) 0.557 ms 0.229 ms 0.202 ms
2 l100.washdc-vfttp-93.verizon-gni.net (173.66.179.1) 6.919 ms 8.588 ms 7.432 ms
3 g1-5-3-0.washdc-lcr-21.verizon-gni.net (130.81.216.76) 12.248 ms 12.530 ms 9.252 ms

So basically Google has figured out a way for its servers to be “close” to me, usually geographically, so I get their content more quickly, or at least with fewer stoplights between their servers and my computer. This magic is done through a content delivery network. I’m pretty sure Google rolled their own, and that takes a lot of money, which Google helpfully has.

You can imagine if a company wanted to create a new amazing search index, it would be at a significant disadvantage if it didn’t have a content delivery network. They probably won’t roll their own like Google, but use one of the companies out there that do this for profit, like Akamai and Level 3. The technology behind this is interesting but I won’t detail it here. The linked Wikipedia article explores it if you are interested. Suffice to say it does not come free, but there are times when it is justified. The U.S. Geological Survey where I work uses a commercial content delivery network. Whenever there is a major earthquake they push the content out to the CDN, otherwise their servers would get overloaded and it would be like a massive denial of service attack. It also gets this data out more quickly to the public, as the typical customer probably only has to traverse three hops instead of thirteen to get the information.

We like to think that the Internet is free, but of course it isn’t. We all pay for access to it. Even if we don’t pay it directly, we pay indirectly, perhaps for the cup of coffee at Starbucks while we surf on their wireless network, or through taxes if we use Internet kiosks at our local library. Doing away with net neutrality is just another means by which ISPs hope to make gobs of money from having a monopoly on the last mile between the content you want and your computer. This may be due, in part, by our refusal to pay for their pricier tiers of service. The only difference is that this time you are not directly paying for it but other content providers will be. (You would think ISPs might cut you in on the deal and discount your rate, but that assumes they are benevolent, and not the profit-obsessed weasels they actually are.) As we all know, nothing is free, so these costs will certainly be passed on to you if you are a subscriber, and that profit will go to the ISP.

Given that bandwidth to the home is a limited commodity, giving discriminatory access to web content providers that can afford to pay must by necessity mean that others will get less access. In that sense, the latest FCC proposal is smoke and mirrors, and it is in everyone’s interest to get off our lazy asses and oppose it.

You can leave a short comment to the FCC here or a long comment here.