ScaleScaleScaleScale

Great Architectures, Stacks & DevOps at Webscale

By Chris Ueland


Rolling Your Own CDN – Build A 3 Continent CDN For $25 In 1 Hour

For a fun weekend project, we decided to roll our own CDN using Go GeoDNS, DigitalOcean, Docker and Nginx.

DIGITALOCEAN

   +      NGINX          +

DOCKER

DigitalOceanNginxDocker

Step 1: Order a DNS instance from DigitalOcean.

We chose SFO and the 2nd smallest instance.

We setup DNS info to point here (both NS1 and NS2 point here for ease of setup).  These are A records.

ns1.deploy.io    192.241.224.116
ns2.deploy.io    192.241.224.116

Step 2: Install Docker and run with “coocood/geodns” image on the DNS instance

See https://index.docker.io/u/coocood/geodns for detail.

Step 3: Order the CDN “edge” instances for Nginx from DigitalOcean.

We used the smallest instance.  We setup DNS for this example using them:

test-sfo-do.deploy.io    107.170.251.75    San Francisco  $5/mo
test-jfk-do.deploy.io    107.170.45.174    New York       $5/mo
test-sin-do.deploy.io    128.199.216.96    Singapore      $5/mo
test-ams-do.deploy.io    188.226.185.173   Amsterdam      $5/mo

Here’s what DigitalOcean’s web interface looks like with all the machines spun up:

CDN edge servers using DigitalOcean

Digital Ocean

SSD Cloud Server, VPS Server, Simple Cloud Hosting

Setup 4: Setup a test domain for the CDN:

If you lookup deployiotest.com the 4 web servers are chosen randomly

If you look up www.deployiotest.com, it will go to the closest geo server.

Set the name servers to point to:

ns1.deploy.io 192.241.224.116
ns2.deploy.io 192.241.224.116

which we setup earlier.

SEE THE CDN GEO DNS WORKING

Using whatsmydns.net (a great tool from Daniel) you can see that we are returning geo enabled results for our test domain “www.deployiotest.com”

CDN GEO DNS test using whatsmydns.net

What’s My DNS?

whatsmydns.net is an online service that allows you to instantly perform a DNS lookup to check a hostnames current IP Address and other DNS information against a selection of random name servers around the world.

{
    "serial":1394372421,
    "ttl": 390,
    "targeting": "country continent @ regiongroup region ip asn",
    "data":{
        "":{
            "ns": ["ns1.deploy.io", "ns2.deploy.io" ],
            "random":[
                [
                    "107.170.45.174",
                    "1000"
                ],
                [
                    "128.199.216.96",
                    "1000"
                ],
                [
                    "107.170.251.75",
                    "1000"
                ],
                [
                    "188.226.185.173",
                    "1000"
                ]
            ]
        },
        "www":{
            "alias": ""
        },
        "www.us-west":{
            "cname": "test-sfo-do.deploy.io."
        },
        "www.europe":{
            "cname": "test-ams-do.deploy.io."
        },
        "www.asia":{
            "cname": "test-sin-do.deploy.io."
        },
        "www.us-east":{
            "cname": "test-jfk-do.deploy.io."
        }
    }
}

Step 5: Verify that your CDN is working using JustPing.com

Just test it on www.just-ping.com.

Todo

:

  1. Note: a comment from akerl_ on HN pointed out “the article doesn’t point out the serious risk of running a single DNS server for the records”. It’s a good point. Think before production – this is intended as an example
  2. Add a way to take servers in and out of rotation using a little daemon on the geodns server
  3. Tune the Geo map for places outside of the US/Europe and Asia.
  4. Add more virtual servers from other providers
  5. Add a way to spin up servers using lib-cloud

Thank you to Ewan Chou with this post.  Ewan is a go developer based out of China.  He likes working on efficient low level code.  His github profile is located here.

Conclusion:

Do you enjoy working on this kind of stuff?  Make it your full time gig and come work with us!

Popular search terms:

  • build your own CDN
  • Build own CDN
  • digitalocean cdn
  • build a cdn
profile

Chris Ueland

http://www.ueland.com

Wanting to call out all the good stuff when it comes to scaling, Chris Ueland created this blog, ScaleScale.

  • there are a bunch of comments @ https://news.ycombinator.com/item?id=8094307 – thanks for reading and all the feedback!

  • Anil

    Nice post and guide though lacks ‘how-to’ steps. Do you have any guidance for sites using SSL? Can they use roll their own CDN for SSL site?

    • hi! you just configure it for SSL in the nginx config. Nginx supports SSL, SPDY and SNI. The config management can be a bit of a pain. You can use any of the opensource stuff or something like http://www.commando.io. At MaxCDN.com we use custom anycast routing and have a whole config layer written in node.js. You can read about it some more here: http://blog.maxcdn.com/improved-api-response-time-95/

      If you want to play w/ a maxcdn account for free let me know and I’ll send you one. chris at maxcdn com

      Thanks for reading the post.

  • csfalcao

    Nice project, DO, NginX and Docker rules!

  • A little correction: Is it 92.241.224.116 or 192.241.224.116?

  • Host http://www.deployiotest.com not found: 2(SERVFAIL) 😉

  • Pingback: Rolling your own CDN for $25 in 1 hour | The Network Nasty()

  • “Step 1: Order a DNS instance from Digital Ocean.”

    I don’t see any one-click installations for DNS servers in DigitalOcean’s applications list. I can tell from the screenshot that it’s ultimately running under CentOS but it would be helpful to have more specific instructions as to how to set that DNS server up.

    • starrychloe

      Yeah, what does it mean ‘Order a DNS instance’? Digital Ocean doesn’t provide ‘DNS instances’! They only provide VPS, where you can put what you want on it. What is ns1.deploy.io ? That’s not Digital Oceans name servers. Their name servers are ns1.digitalocean.com, ns2.digitalocean.com, and ns3.digitalocean.com.

      • I think what they mean is they set up their own nameserver using DNS server software on a digital ocean droplet. But they don’t say what software, or how they set it up. Some more instructions would go a long way!

        • hey, good feedback! I didn’t expect this to get so popular and this was a basis I did for a 10 minute docker presentation. I’ll see if we can add some more details.

      • bad_ip

        This is actually clearly made to present an architectural concept and assumes you understand how to install and maintain regular Linux infrastructure.

  • Pingback: CDN на практике - Linux в Беларуси()

  • Ben

    I love the idea. however, how would you do content caching/replication at each edge site?

  • Pingback: July Top 10 links | HighOps()

  • Please forgive me my ignorance. How would you combine this (GeoDNS) *with* Ngnix/Varnish on two or three DigitalOcean droplets?

    I now have one DigitalOcean droplet with Nginx and Varnish in the US for content offloading to, and caching in the US. I’d like to expand to SE Asia with a second droplet, and use a GeoDNS solution to offload to the nearest CDN hostname. Syncing the content between droplets should be possible with lsyncd I guess.

  • bad_ip

    You would use an application called Varnish.

  • BeachWebDesigner

    Good Stuff. Cheers.

  • Dennis Felyx

    Great article!!!!

  • it’s working if end user using opendns? I have try with dns split horizon & HAproxy geoip. but not working if end user using opendns like google etc

  • Mark Caldwell

    This Web page is screwed.

    1. The link to docker fails and lands on a docker page with a meta refresh of 0. You have to hit the back button repeatedly and super fast to get back. I guess the folks at docker have an idiot webmaster also.

    2. Why would it ask for capita on returning.

    3. Then the capita fail I am guessing and block you from the site. Saying I am up to mischief when all I want is to read the page.

    4. Also note title says build for $25 and your dns instance alone is $20 or $30. Plus 4x $5 for the edge .

    I’m sure I registered as many hit to this page, yet was allowed to read half before some Rouge software blocked me. So while you think I read the page. The reality is, that regardless of how good the written content may have been, I stopped right here and moved on to find a page that does not completely suck.

  • i don’t understand anything here