When you host resources (web, database, forums) for the public you probably want to make sure you know what to do when something breaks. Most of the time that means performance and availability issues, but sometimes it means compliance issues (legal, ethical) or migration (from one provider to the next). The earlier in the process you separate what’s under your control the better, but it’s important to note that having more control is not always the best.
For the last few years, in my spare time, I am refining a process I call “self-hosting”. I go far and deep beyond hosting a web or a wiki - I start at the infrastructure foundation, which is the DNS service. Two of my most favorite solutions (Cloudmin and YunoHost) go this deep - they install and use a DNS server and a dedicated (sub)domain.
I do not want to waste anyone’s time reading this, so what I wrote so far should make it pretty clear if you have any use for a fully mapped internet infrastructure. Unless you are tasked at work with creating one - unless you are training to be able to create and maintain one - unless you are teaching others to be able to create one - you probably do not need to read any further. I fall into the category of a person who is self-training, all I (assume) to know about this subject is from self-training.
The scope and the purpose of this post is to illustrate a fully configured, 100% internet standards compliant infrastructure. It is going to show you how to separate what you “want” to control, you are “able” to control, you “should” control, from the rest. You do not need to follow and implement what I show here to understand key concepts. Please critique without worry about hurting my feelings, I am very grateful for everyone who donate their time and effort. Time is the most precious resource we have, so let’s not waste any more of it.
. . . A proper DNS infrastructure requires two servers: one Primary (let’s call it NS1) and one Secondary (NS2). You could have more, but unless you host many domains world-wide more than two only adds extra work and expense, without any benefits.
. . . Everyone depends on three different entities for DNS. This is illustrated by the graphical mapping http://dnsviz.net/d/factoryfouroh.dev/dnssec/ which is one of my own zones (I am still building NS2, and you will see that for a while). It shows the chain of authority from the root, to the “dev” Top Level Domain (TLD), to the “factoryfouroh” Zone.
. . . a) Nobody controls the root, the official standards body does what it must do to keep it working, to protect it from attacks, to prevent it from total collapse. The root controls the internet the same way our bodies demands us to eat, to rest, to breath.
. . . b) Your registrar controls your access to the TLD, the only place you may change glue records, point IP addresses to Start of Authority (SOA) records, and add encryption keys for most but not (yet) all TLDs.
. . . c) The actual name resolution service and its database, the DNS, may be fully under your control - or you may trust someone else to manage it for you. You may consolidate different TLDs (register all your .com, .net. .org) at the same Registrar - but there is no way to ‘consolidate’ the three entities in the chain of authority. Out of the three you may control a maximum of one, even when you are a Registrar like GoDaddy or Google.
. . . Hosting a DNS server requires minimal resources (time, money, knowledge), but it is important to understand that most Internet standards and best practices are created and updated to be followed with religious fanaticism! Look at http://dnsviz.net/d/gmail.com/dnssec/ and see how Google maintains its infrastructure. If your DNS provider does not deliver the same you are significantly shortchanged. Chances are you did not do all you should, you may ask your DNS provider (which might or might not be the same as your Registrar) to help you, or you may host your own - but you should NOT ignore it.
. . . I prefer to host NS2 after all these considerations.
. . . . a) Google Computing Engine (GCE) and Linode are both excellent choices for cost and very quick backup / restore cycles. Some providers do not offer backup - do not use them for NS2.
. . . . b) Putting your DNS behind the hosting provider’s firewall is a huge value! It takes some work to figure out AWS and Google firewalls, but once you make it work for your purposes you may rest assured.
. . . . c) Reverse DNS is required for NS2 by DNSSEC, both Google and Linode provides this with the IP4. With rDNS in place you could, in theory, also install a send-only email service you may use for all sorts of purposes such as monitoring or sending user registration and password resets - but should not make available for normal user logins. I admit, so far I used NS1 for this, but in the future I am going to move the email to NS2.
. . . . d) Your NS2 is the really important instance. With NS1 (and when you host the SOA on NS1) you cannot help but edit records and restart, sometimes restore or may even rebuild! Especially when you are a novice you do all sort of stupid stuff. So make sure NS2 is there and its running.
. . . . e) To create NS2 I usually install a Debian 9 instance and do this (reboot as necessary, but at least once before autoclean):
apt-get update apt-get -y install nano apt-transport-https tzdata apt-get -y install locales locale-gen "en_US.UTF-8" dpkg-reconfigure tzdata wget http://www.webmin.com/jcameron-key.asc apt-key add jcameron-key.asc apt-get update apt-get -y install webmin apt-get -y autoclean apt-get -y autoremove
. . . . g) Put this behind a firewall profile with all ports but ssh, https, dns closed. You may harden it further by disabling root login, changing the ssh port, etc - but this is the minimum. I also open port 10000 for Webmin, or the range 10000-10100 when I run Cloudmin.
. . . h) I do nearly everything else from the Webmin interface, including installing and configuring DNS. If you know what you are doing you may do it all from the shell, including backing up and restoring the zones folder.
. . . For NS1 add most there is for the Secondary. Since Google does not allow email traffic from GCE instances I use a 28 / UDS a year CheapVPS instance. You need to ask for the rDNS - but they will give you one if you ask. Of course you smile, the great NS1 should be something more substantial, no? The answer is no, your NS2 is the one that should be mightier.
Webmin makes configuring DNS very self-explanatory, and if there is demand I am going to detail it in another post, maybe even with some nice screenshots. In this post I am only listing what is configured - controlled by the DNS.
. . . . a) Encryption is essential, and maybe - just maybe - in the near future there is going to be a way (or easier way) to become your own Certificate Authority (CA). Until then you still need to maintain a Public key Infrastructure (PKA), and Webmin makes this really, really painless.
. . . . b) To send email to gmail accounts your email service must be configured flawlessly, with 100% standards compliance - otherwise gmail going to send them to Spam. I use YunoHost on a subdomain to serve email, as YunoHost is a dream to install and configure! After installation you add all DNS record YunoHost prints for you and you have valid, secure email.
. . . . c) To manage clusters of domains and zones the GPL version of Cloudmin / Virtualmin creates and manages all KVM instances and DNS records. The subscription does much, much more allowing you to grow a cloud pretty big and complex, with all sort of virtualization for Virtualmin hosts (KVM, Docker, OpenVZ etc… with AWS, GCE, Xen hooks, replication, failover, too many to list). Since I never managed to make DKIM work on Virtualmin I do not use it for email, and the reason I never figured out is because YunoHost is sooo awesome! Yet YunoHost is buttoned down pretty tightly where Cloudmin/Virtualmin is just the best!
As I consider taking part in High Fidelity I could not help to post this here for you for my own reference, and for those interested. Anything worth doing should be done properly, and I am not at all comfortable running a server for years to come out of my home. I am going to put that server on the internet, and I am going to start by creating a sub-domain for it under my full control. I am going to try and use this thread to document the process.
Thank you for reading.