VPS Setup: FQDN DNS Setup With GoDaddy

This is a subpost of the larger post Updated Comprehensive VPS Setup Documentation.

 

Once you have a VPS deployed and know its static IP, you can forward a FQDN to it by creating a new A Record. I use GoDaddy for my DNS registration because they are simple, reliable, and quick.

  1. In order to do this with GoDaddy, log into your account
  2. Next to “Domains” click on “Manage”
  3. Click on the domain you want to forward. If you want to forward a subdomain, click on the domain it will be a subdomain of
  4. Click over to the “DNS ZONE FILE” tab
  • If you are trying to forward a subdomain
    1. Click “Add Record”
    2. We want to create an “A Record”
    3. Use the subdomain as the hostname and then the static IP of the server as the “POINTS TO”
    4. Save changes
  • If you are trying to forward a domain
    1. Edit the record for the “@” host and point it to the static IP following the same directions as above.

VPS Setup: Deploying A New Virtual Private Server With Digital Ocean

This is a subpost of the larger post Updated Comprehensive VPS Setup Documentation.

 

I like Digital Ocean (Referral Link) for my VPS host.

The first step in creating a new VPS is to select a Linux distribution. I always use the most current version of Debian. At the time of this post, that is version 8.3 x64.

Decide on an FQDN and use it as the hostname and hit deploy!

It should take about a minute and then you will receive an email with the temporary root password. Use putty to log in, and you will be prompted to change it.

Make sure to choose a root passsword with a high level of entropy.

Building a Jabber/XMPP Server With OpenFire and Debian

I wanted to create a new chat server for my company for two reasons. For one, we want any confidential information to stay as in-house as possible. And two, we wanted web access because lots of our employees move around between different offices and they don’t want to have to install chat programs every day.

We had previously been using OpenFire hosted on a local baremetal machine which did not have a CA signed cert. This meant we could not use OpenFire’s web access tool in order to access the chat tool because it did not support self-signed certs. Trillian could be talked into supporting self-signed certs, but a more elegant solution was called for.

I decided to create a new jabber/xmpp VPS with DigitalOcean (Referral Link) and install OpenFire and SparkWeb.

  1. Step one was creating a new VPS or “Droplet” with DigitalOcean (Referral Link) I chose Debian x64 for the OS and used my new FQDN for the hostname.
  2. I created a new DNS “A Record” with our hosting provider to forward my new FQDN to this new server’s IP.
  3. Install all the initial stuff. This should be self-explanatory: apt-get -y update && apt-get -y upgrade && apt-get -y install default-jre fail2ban apache2 && apt-get -y install mysql-server && apt-get -y install php5 php-pear php5-mysql && apt-get -y install php5-mcrypt && php5enmod mcrypt && a2enmod rewrite && apt-get -y install php5-curl phpmyadmin screenfetch htop nload curl git ntp && service apache2 restart && mysql_secure_installation
  4. Navigate to the downloads page http://www.igniterealtime.org/downloads/index.jsp and find the path to the Debian installer file of the current version
    1. Get the installer with wget -O openfire_installer.deb [LINK], replacing [LINK] with the link from the page in the previous step
    2. In my case, it was wget -O openfire_installer.deb http://www.igniterealtime.org/downloadServlet?filename=openfire/openfire_4.0.1_all.deb
  5. Install OpenFire with dpkg --install openfire_installer.deb
  6. Block insecure access to the OpenFire admin console with iptables -A INPUT -i eth0 -j REJECT -p tcp --dport 9090
  7. Install LetsEncrypt for free SSL:
    1. Now that OpenFire is configured, navigate to the root user’s home directory and clone letsencrypt with git clone https://github.com/letsencrypt/letsencrypt
    2. Enter the directory cd letsencrypt
    3. And run the automatic script ./letsencrypt-auto --apache
    4. It will ask which virtual hosts you want to install certificates for, and then it does all the work for you!
  8. Navigate to https://[FQDN]:9091 and complete the configuration

 

UPDATE 2016-07-31

We have officially migrated to Slack as a company. This provides compliance with all the various requirements of our many managed services clients. At long last, this service has been outsourced to a competent partner, and it is one less thing we need to worry about!

Nevertheless, this guide will show you how to create a simple, free alternative with Jabber/XMPP.

Virtualizing An Application Server

Another department at Tech 2U performs diagnostics on lots of computers. They use a proprietary tool that they built which deploys diagnostic tools on customers’ computers during tech support services.

This tool was built years ago by someone who no longer works here. He used a baremetal Apache server to host the tools. This server crashed, crippling the tool and everyone who relied on it.

I decided to move the tools to a new cloud VPS.

I created a new Droplet on Digital Ocean (Referral Code) for $5/month.

I chose Debian 8 amd64 for the OS and set a hostname of the new fqdn.

Once I created the droplet, I pulled its IP and created a new DNS A-Record on our main domain account to forward that hostname to the new VPS.

 

Now the VPS was finished building so I ran apt-get -y update && apt-get -y upgrade to update any packages that have changed since Digital Ocean built their image for this type of server.

apt-get -y install fail2ban apache2 && apt-get -y install && a2enmod rewrite && service apache2 restart && apt-get -y install screenfetch htop nload curl git ntp

Now I make a new folder for the virtualhost with mkdir /var/www/[fqdn]

Now we make a new virtualhost conf file with this command. Again, substitute your fqdn;

cp /etc/apache2/sites-available/000-default.conf /etc/apache2/sites-available/[fqdn].conf

Then edit the file with nano /etc/apache2/sites-available/[fqdn].conf

It needs to contain the following;

	ServerName [fqdn]

	ServerAdmin your_email@website.com
	DocumentRoot /var/www/[fqdn]/

	ErrorLog ${APACHE_LOG_DIR}/error.log
	CustomLog ${APACHE_LOG_DIR}/access.log combined

I also created another virtualhost to server the /var/www directory. This virtualhost will be secured with a directory password and contain some diagnostic and performance monitoring tools.

Now I disable the default VirtualHost with a2dissite 000-default

Then enable my new virtualhosts with a2ensite [fqdn]

And restart Apache with service apache2 restart

BitTorrent Sync

If you’re not already familiar with BitTorrent Sync, it is a free, secure option for synchronizing directory structures in real time. I use it to synchronize the app between different web servers. Any changes are immediately propagated everywhere. This is also the vehicle for delivering updates to the server from the people who manage it.

This command will download the script to install version 1.4 of BitTorrent sync. Note that this is not the most recent version, as the new version is very limited in features and requires much more resources to run.
sh -c "$(curl -fsSL http://debian.yeasoft.net/add-btsync14-repository.sh)"

Then, run this command to install BitTorrent Sync
apt-get update && apt-get install btsync

Now we need to clean up any permissions and ownership issues with the following commands;

chmod 775 /var/www/ -R
chown www-data:www-data /var/www/ -R
chown www-data:btsync /var/www/f2.tech2u.com -R

Then we configure btsync to synchronize the app’s folder, and it does the work of importing the app.

Now I simply add the old BitTorrent key to the correct directory and then all the files copy over!

 

Eureka!

screenfetch

Moving My App To The Cloud

I have spent the better part of the last three years building a scalable logistics platform which has grown to manage nearly all the daily operations of my workplace, a mobile tech support company.

Some Major Features;

  • Booking appointments and scheduling them for employees in different regions and markets
    • Providing a portal for employees to see their jobs and communicate status to central dispatch
  • Handling accounting
    • Lots of custom reporting and alerts built in
  • Payroll calculations
    • Lots of different pay structures and commission rates
  • Reporting on sales data
    • Accounting and strategy development
    • Employee development
  • Outreach
    • Automatic outbound emails and calls for lots of different segmentations and purposes
    • Outbound sales for well-qualified customers
    • Follow-up surveys for quality assurance and customer satisfaction reporting
  • Business Intelligence
    • Predictive algorithms identify logistical problems before they happen and suggest fixes

This app started out as a way to make my job easier, managing daily operations and logistics. It quickly grew to take over much of our operations management, facilitating daily operations across the country and automating many previously tedious procedures and functions.

As it grew, it became more mission-critical.It currently provides essential infrastructure to many employee roles and daily operations.

A recent series of power outages proved that hosting it in-house was a bad idea, so I have begun to work on a solution: migration to the cloud.

Let’s Get Started

  1. The first step was to build a VPS using my Setting Up WordPress on a VPS tutorial to build the VPS, and then instead of installing WordPress, I copied over my app’s PHP files from its previous server.
  2. Next, I needed a secure connection to the existing databases until they can be migrated as well.
    I considered using a VPN, but this would be unduly complex because of the way our corporate network is setup.SSH is a great alternative to VPN.

    I used SSH to create a secure tunnel from the office to the new VPS. SSH has a feature which allows you to bind a port on the local machine to a port on the remote machine. In this way, I was able to connect from the VPS to the SQL server in the office.

  3. Next, I needed to rebuild many aspects of my app to use locally cached data instead of remote data. I built a new tool which continuously synchronizes a local copy of the remote database. This means that the remote database connection can fail without affecting the app. This parallels the future use-case when many companies may use the app, and all their data would need to be synchronized into the local database as well.

Setting Up WordPress on a VPS

This post goes over how to setup WordPress on a VPS. This is sort-of a cumulative post which incorporates lots of little tutorials I’ve done and lots of new best-practices I have learned through trial and error.

If you’re following my posts in general, you may have noticed that I already did this. I made a mistake while using a beta tool which caused me to need to start over, and my many posts documenting the processes had become fragmented so I figured it was time to do a sort of roll-up mega-post.

Step One: Build LAMP Stack

The first step is to create a VPS. I use DigitalOcean (Referral Link for two months free) because it is cheap, reliable, and easy to use. It takes less than a minute to create a VPS and there are lots of OS options. In this case I chose the most current version of Debian, Version 8.6 amd64.

Once the vm is set up, we need to connect via SSH. I am using Windows 10, so I will be using putty which is a great, free option. Once you setup a VPS, DigitalOcean will send you an email with your ip and credentials. Use those to log in via SSH.

Right out of the box, we need to make sure everything is up to date on the new machine;

apt-get update && apt-get upgrade

The first thing I like to do is install fail2ban so that I know no one can bruteforce my SSH password.

apt-get install -y fail2ban

Easy enough. The next step is to install Apache.

apt-get install -y apache2

Now install and configure MySQL. Make sure to pick a good password and only allow connections from localhost when it asks.

apt-get -y install mysql-server
mysql_secure_installation

Next we install PHP

apt-get -y install php5 php-pear php5-mysql

As well as a few add-ons that our tools will need;

apt-get install php5-mcrypt
php5enmod mcrypt
a2enmod rewrite
apt-get install php5-curl

If you are planning on using any MSSQL connections, use this;
apt-get install freetds-common freetds-bin unixodbc php5-sybase

Restart Apache to make everything take effect…
service apache2 restart

Now create a test page by opening a new file in the web root:

nano /var/www/info.php

Add the following code and Ctrl-X to exit.

<?php echo phpinfo(); ?>

If you see something like this, we are done setting up the LAMP stack and ready to move on…

phpinfo

Step Two: Setup Email

Postfix is a great, free option that enables the server to send emails reliably.

The first step is to block non-local requests to send emails.

iptables -A INPUT -i eth0 -j REJECT -p tcp --dport 25

Then install Postfix and associated tools:

apt-get -y install postfix
apt-get -y install mailutils

Now edit the config files and change the interface to loopback-only like so;

nano /etc/postfix/main.cf

Find this line;

inet_interfaces =

And change to;

inet_interfaces = 127.0.0.1

Now edit the email aliases;

nano /etc/aliases

At the end of the file, make sure there is a line that starts with root and ends with your email, like so;

root email@domain.com

Save the file and exit. Then run newaliases to let Postfix apply the changes.

Restarting Postfix is not enough because we changed the interfaces line in the config file. We need to stop and start it like so;

postfix stop
postfix start

Step Three: Create New VirtualHost

First lets create a new directory for our install. Substitute your fqdn;

mkdir /var/www/[fqdn]/

Now we make a new virtualhost conf file with this command. Again, substitute your fqdn;

cp /etc/apache2/sites-available/000-default.conf /etc/apache2/sites-available/[fqdn].conf

Then edit the file with nano /etc/apache2/sites-available/[fqdn].conf

It needs to contain the following;

<VirtualHost *:80>
	ServerName [fqdn]

	ServerAdmin your_email@website.com
	DocumentRoot /var/www/[fqdn]/

	ErrorLog ${APACHE_LOG_DIR}/error.log
	CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Activate the new virtualhost with a2ensite [fqdn]

Restart apache with service apache2 restart

Step Four: Setup WordPress

Download the latest version of WordPress from their website. Then upload the files to the appropriate directory which we created above. Then we need to change the ownership and permissions of the directory;

chown www-data:www-data /var/www/ -R
chmod 775 /var/www/ -R

Navigate to the web url, and you will be guided through the setup process to get wordpress connected to the database we setup earlier.

Migrating Everything to a Cloud VPS

Over the last six months, I undertook the project of migrating all of my web apps, sites, and other projects to a new XenHypervisor which I hosted at home for free.

I always thought of this as a transitional step. It doesn’t make sense to try to host these kinds of things at home. After all, I don’t want to be a datacenter provider, but I wanted to understand how they work before I outsourced my infrastructure to a good provider.

digital-ocean-logo

After lots of research, I decided on DigitalOcean (Referral Link) as the best option. It’s only $5/month and I don’t have to worry about infrastructure or connectivity issues.

  • Setting up the VPS was easy and took just a few seconds. DigitalOcean makes it very simple to create the new machine, and they had the latest version of Debian as an option.
  • Next, I setup an A-Record for (vps1.cjtrowbridge.com) a new subdomain off of my main domain. This serves as a FQDN for this new VPS. (Note that this is my second VPS, I am not using a 1-based index 😛 )
  • Referencing my previous tutorial about Setting Up Debian as a LAMP Server, I configured the new machine with all the normal tools.
  • I created virtual hosts for all the sites I was migrating in and copied those files and databases over to the new VPS.
  • I used this tutorial to secure the root web directory for the VPS’ FQDN. This directory contains all the directories for the virtual hosts and tools lime PHPMyAdmin, as well as the backups for other servers. With valid credentials, this lets you see all the tools, hosts, and backups.
  • Lastly, I setup BitTorrent Sync using this tutorial in order to securely backup files across the internet.

Building the Web Application Framework Stack

This post begins at the point of having installed Debian, Apache, MySQL, and PHP and being ready to start developing web applications.

Now I need to install the technologies that facilitate some of the higher level features I will be using.

Installing and Configuring Git

First step is to install Git. This will help keep track of versioning and changes as the work progresses.

apt-get -y install git

I followed these steps to configure my local git account;

Then I setup the repository for this project by navigating to the correct directory, in this case…

cd /var/www/projects

And running the command;

git init

I am greeted with the message “Initialized empty Git repository in /var/www/projects/.git/”

Installing Node.Js

The first step is to get the install file;

curl --silent --location https://deb.nodesource.com/setup_0.12 | bash -

Then run the install;

apt-get install nodejs

Installing Bower

Bower is a package manager for web application frameworks that helps keep all our frameworks up to date, along with their dependencies. I installed it with this command;

npm install -g bower

In order to configure bower, the following commands need to be run as a user instead of root, so I used su cj to switch to my user account. And then I installed some frameworks;

bower install jquery
bower install jquery-ui
bower install bootstrap
bower install angular
bower install material-design-icons
bower install polymer

If you encounter permissions errors at this step, you may need to use something like chown -R cj:cj /var/www to give yourself ownership of the directories. I created new virtual hosts for a number of different websites I am hosting on this server, including this blog. As such, they were owned by the root account that created them, and bower was not able to access them from the user account until I made my user account the owner.

Now all I need to do in order to keep every part of the server and all my frameworks up to date is periodically run;

apt-get update && apt-get -y upgrade && npm update && bower --allow-root update

Building a XenServer at Home with a Local Repository

I have gone over the process of setting up a XenServer before, but this time there is one major difference. It will be using a local repository instead of using a NAS as an ISO repository.

This was a little tricky but not too bad.

Once I had XenServer installed and configured, I opened an SSH session on the server.

I used the following commands to create a local repository;

mkdir -p /var/opt/xen/ISO_Store

xe sr-create name-label=LocalISO type=iso device-config:location=/var/opt/xen/ISO_Store device-config:legacy_mode=true content-type=iso

Now when I open XenCenter, there was a local repository attached to the server. I renamed the repository form within XenCenter to be more clear.

local_repository

Now I can use wget to download files to the repository like so;

cd /var/opt/xen/ISO_Store

wget http://cdimage.debian.org/debian-cd/8.2.0/amd64/iso-cd/debian-8.2.0-amd64-CD-1.iso


 

Now I can create a new vm and use this local ISO! Once the download is complete, the file will be visible in the list;

 
debian_local

Migrating our Enterprise Production Environment From ESXi to Xen

When I started developing software at my current workplace, our web app server was running Turnkey Linux on Debian 6 inside a hypervisor running VMware ESXi 4.5; all of these tools were already very obsolete when I joined the development environment, and after over two years, it was time to make some changes.

debianI decided to move to a modern, open source hypervisor and the current version of Debian without the hundreds of irrelevant packages that come with Turnkey Linux.

I started by creating a new VM and installing Debian 8 with Apache, PHP and MySQL and then I migrated all the PHP scripts and the database over from the old web app VM to this new virtual server.

Now it was time to create a temporary hypervisor while I upgrade the current physical server. I found a disused workstation from the office and installed XenServer on it.

Both VMWare and Xen have a feature where you navigate to the server’s IP address in a browser and they give you a download for the desktop management tool to let you work with the server. Installing these was simple enough, then I exported all the old ESXi hypervisor’s virtual machines to OVF files, a universal standard format for virtual machine migrations.

Importing them into XenServer took A LONG TIME; I ended up leaving it overnight. But there were no problems on any of the VMs; Windows or several flavors of Linux, everything went very smoothly.

With the new VMs successfully imported, I shut down the old VMs and Started them new ones up. They took over their same static IPs and booted up as though nothing had happened.

virtualization-missingNow it was time to upgrade the old VMWare ESXi server to Xen. This process was a lot harder than I expected, and I did encounter two problems trying to get it to boot up for the first time.

Because it was an older Dual-Xeon server, it was missing an architecture feature required for some types of virtualization. It said Windows VMs might potentially have issues. This is fine for me because I am not using any Windows VMs. I looked at Dell’s website and there are no BIOS updates currently available which appear to resolve this issue. It may be impractical to use an older Dell server like this for hosting windows machines with Xen.

panicAnother problem arose because the older hyperthreading architecture threw a non-maskable interrupt parity error which caused a Kernel Panic and halted the machine. It took hours of research and work to solve this problem.

The first step was to disable hyperthreading in bios. Dell couldn’t make this simple; my server’s bios referred to it as disabling the “Logical Processor Feature.” Now the server was able to boot up, though it was only using one core on just one of its CPUs, instead of all the cores on both Xeons.

In order to prevent this Kernel panic, we need to tell the system to skip parity checks for NIM (or non-maskable interrupts). This is theoretically simple enough, but finding the bootloader configuration file proved very difficult as it was not in any of the normal places.

Once in the “Console” section of the XenCenter tool, the next step was finding the bootloader configuration file. This may differ for different versions of Xen, mine is 6.5. I eventually found the file here;

 /boot/extlinux.conf 

Find the section that looks like this…

label xe
  # XenServer
  kernel mboot.c32
  append /boot/xen.gz dom0_mem=752M,max:752M watchdog 

We need to add “nmi=ignore” into the “append” section like so;

label xe
  # XenServer
  kernel mboot.c32
  append /boot/xen.gz dom0_mem=752M,max:752M watchdog nim=ignore

After saving this file, I was able to reboot and turn the “Logical Processor Feature” back on, enabling all the cores and CPUs in the server. Then I was rewarded with a happy boot screen;

booting-up

Moving the virtual machines back to this server was as simple as moving them to the temporary one. Now everything is setup and running happily on this new, modern and open source hypervisor! 😀