VPS Setup: Create A Virtual Host

This is a subpost of the larger post Updated Comprehensive VPS Setup Documentation.

 

Once you have your FQDN forwarded to the VPS, create a directory for it with;

mkdir /var/www/[fqdn]/

Now we make a new virtualhost conf file with this command. Again, substitute your fqdn;

cp /etc/apache2/sites-available/000-default.conf /etc/apache2/sites-available/[fqdn].conf

Then edit the file with nano /etc/apache2/sites-available/[fqdn].conf

It needs to contain the following;

	ServerName [fqdn]

	ServerAdmin your_email@website.com
	DocumentRoot /var/www/[fqdn]/

	ErrorLog ${APACHE_LOG_DIR}/error.log
	CustomLog ${APACHE_LOG_DIR}/access.log combined

Activate the new virtualhost with a2ensite [fqdn] and if you haven’t already done this, deactivate the default virtualhost with a2dissite 000-default.conf

Restart apache with service apache2 restart so the changes take effect.

 

Automated Backups

If you want to setup automated backups, create a new directory for the backups;

mkdir /var/backups/[fqdn]

 

Add the following line to /etc/crontab in order to facilitate automatic daily backups;
0 22 * * * root tar -cf /var/backups/[fqdn]/www-backup-$( date +'\%Y-\%m-\%d_\%H-\%M-\%S' ).gz /var/www/[fqdn]

 

Or if you would prefer weekly updates every Sunday night, use this instead;

0 0 * * 0 root tar -cf /var/backups/[fqdn]/www-backup-$( date +'\%Y-\%m-\%d_\%H-\%M-\%S' ).gz /var/www/[fqdn]

VPS Setup: Email Server

This is a subpost of the larger post Updated Comprehensive VPS Setup Documentation.

 

Many of my apps send lots of emails, so I usually need to setup a local outbound email server.

Secure the port with iptables -A INPUT -i eth0 -j REJECT -p tcp --dport 25

Install postfix for the server apt-get -y install postfix && apt-get -y install mailutils

Now edit the config files and change the interface to loopback-only like so;

nano /etc/postfix/main.cf

Find this line;

inet_interfaces =

And change to;

inet_interfaces = 127.0.0.1

Now edit the email aliases;

nano /etc/aliases

At the end of the file, make sure there is a line that starts with root and ends with your email, like so;

root email@domain.com

Save the file and exit. Then run newaliases to let Postfix apply the changes.

Restarting Postfix is not enough because we changed the interfaces line in the config file. We need to stop and start it like so;

postfix stop
postfix start

VPS Setup: Recommended Initial Installations

This is a subpost of the larger post Updated Comprehensive VPS Setup Documentation.

 

When initially setting up a VPS, I generally install the programs listed below. Before installing anything, it is important to first update and upgrade all packages already installed on the server with apt-get update && apt-get upgrade

  1. First, install Fail2Ban in order to prevent bruteforcing of SSH passwords
  2. Install Apache2
  3. Install MySQL Server
  4. Install PHP and its dependencies for MySQL and PHPMyAdmin
  5. Performance Tools
    1. Screenfetch lets you see system information
    2. Htop lets you see details about resource usage
    3. Nload lets you see details about network utilization
  6. NTP makes sure the time is kept up to date
  7. Git tracks changes in files and is required for LetsEncrypt

This command will do all of these things without prompting in between;

apt-get -y install fail2ban apache2 && apt-get -y install mysql-server && apt-get -y install php5 php-pear php5-mysql && apt-get -y install php5-mcrypt && php5enmod mcrypt && a2enmod rewrite && apt-get -y install php5-curl && service apache2 restart && mysql_secure_installation && apt-get -y install phpmyadmin && apt-get -y install screenfetch htop nload curl git ntp

You will be prompted to create a MySQL root password. PHPMyAdmin setup will ask you for this password, as will the MySQL Secure Installation tool.

VPS Setup: FQDN DNS Setup With GoDaddy

This is a subpost of the larger post Updated Comprehensive VPS Setup Documentation.

 

Once you have a VPS deployed and know its static IP, you can forward a FQDN to it by creating a new A Record. I use GoDaddy for my DNS registration because they are simple, reliable, and quick.

  1. In order to do this with GoDaddy, log into your account
  2. Next to “Domains” click on “Manage”
  3. Click on the domain you want to forward. If you want to forward a subdomain, click on the domain it will be a subdomain of
  4. Click over to the “DNS ZONE FILE” tab
  • If you are trying to forward a subdomain
    1. Click “Add Record”
    2. We want to create an “A Record”
    3. Use the subdomain as the hostname and then the static IP of the server as the “POINTS TO”
    4. Save changes
  • If you are trying to forward a domain
    1. Edit the record for the “@” host and point it to the static IP following the same directions as above.

VPS Setup: Deploying A New Virtual Private Server With Digital Ocean

This is a subpost of the larger post Updated Comprehensive VPS Setup Documentation.

 

I like Digital Ocean (Referral Link) for my VPS host.

The first step in creating a new VPS is to select a Linux distribution. I always use the most current version of Debian. At the time of this post, that is version 8.3 x64.

Decide on an FQDN and use it as the hostname and hit deploy!

It should take about a minute and then you will receive an email with the temporary root password. Use putty to log in, and you will be prompted to change it.

Make sure to choose a root passsword with a high level of entropy.

Building a Jabber/XMPP Server With OpenFire and Debian

I wanted to create a new chat server for my company for two reasons. For one, we want any confidential information to stay as in-house as possible. And two, we wanted web access because lots of our employees move around between different offices and they don’t want to have to install chat programs every day.

We had previously been using OpenFire hosted on a local baremetal machine which did not have a CA signed cert. This meant we could not use OpenFire’s web access tool in order to access the chat tool because it did not support self-signed certs. Trillian could be talked into supporting self-signed certs, but a more elegant solution was called for.

I decided to create a new jabber/xmpp VPS with DigitalOcean (Referral Link) and install OpenFire and SparkWeb.

  1. Step one was creating a new VPS or “Droplet” with DigitalOcean (Referral Link) I chose Debian x64 for the OS and used my new FQDN for the hostname.
  2. I created a new DNS “A Record” with our hosting provider to forward my new FQDN to this new server’s IP.
  3. Install all the initial stuff. This should be self-explanatory: apt-get -y update && apt-get -y upgrade && apt-get -y install default-jre fail2ban apache2 && apt-get -y install mysql-server && apt-get -y install php5 php-pear php5-mysql && apt-get -y install php5-mcrypt && php5enmod mcrypt && a2enmod rewrite && apt-get -y install php5-curl phpmyadmin screenfetch htop nload curl git ntp && service apache2 restart && mysql_secure_installation
  4. Navigate to the downloads page http://www.igniterealtime.org/downloads/index.jsp and find the path to the Debian installer file of the current version
    1. Get the installer with wget -O openfire_installer.deb [LINK], replacing [LINK] with the link from the page in the previous step
    2. In my case, it was wget -O openfire_installer.deb http://www.igniterealtime.org/downloadServlet?filename=openfire/openfire_4.0.1_all.deb
  5. Install OpenFire with dpkg --install openfire_installer.deb
  6. Block insecure access to the OpenFire admin console with iptables -A INPUT -i eth0 -j REJECT -p tcp --dport 9090
  7. Install LetsEncrypt for free SSL:
    1. Now that OpenFire is configured, navigate to the root user’s home directory and clone letsencrypt with git clone https://github.com/letsencrypt/letsencrypt
    2. Enter the directory cd letsencrypt
    3. And run the automatic script ./letsencrypt-auto --apache
    4. It will ask which virtual hosts you want to install certificates for, and then it does all the work for you!
  8. Navigate to https://[FQDN]:9091 and complete the configuration

 

UPDATE 2016-07-31

We have officially migrated to Slack as a company. This provides compliance with all the various requirements of our many managed services clients. At long last, this service has been outsourced to a competent partner, and it is one less thing we need to worry about!

Nevertheless, this guide will show you how to create a simple, free alternative with Jabber/XMPP.

Virtualizing An Application Server

Another department at Tech 2U performs diagnostics on lots of computers. They use a proprietary tool that they built which deploys diagnostic tools on customers’ computers during tech support services.

This tool was built years ago by someone who no longer works here. He used a baremetal Apache server to host the tools. This server crashed, crippling the tool and everyone who relied on it.

I decided to move the tools to a new cloud VPS.

I created a new Droplet on Digital Ocean (Referral Code) for $5/month.

I chose Debian 8 amd64 for the OS and set a hostname of the new fqdn.

Once I created the droplet, I pulled its IP and created a new DNS A-Record on our main domain account to forward that hostname to the new VPS.

 

Now the VPS was finished building so I ran apt-get -y update && apt-get -y upgrade to update any packages that have changed since Digital Ocean built their image for this type of server.

apt-get -y install fail2ban apache2 && apt-get -y install && a2enmod rewrite && service apache2 restart && apt-get -y install screenfetch htop nload curl git ntp

Now I make a new folder for the virtualhost with mkdir /var/www/[fqdn]

Now we make a new virtualhost conf file with this command. Again, substitute your fqdn;

cp /etc/apache2/sites-available/000-default.conf /etc/apache2/sites-available/[fqdn].conf

Then edit the file with nano /etc/apache2/sites-available/[fqdn].conf

It needs to contain the following;

	ServerName [fqdn]

	ServerAdmin your_email@website.com
	DocumentRoot /var/www/[fqdn]/

	ErrorLog ${APACHE_LOG_DIR}/error.log
	CustomLog ${APACHE_LOG_DIR}/access.log combined

I also created another virtualhost to server the /var/www directory. This virtualhost will be secured with a directory password and contain some diagnostic and performance monitoring tools.

Now I disable the default VirtualHost with a2dissite 000-default

Then enable my new virtualhosts with a2ensite [fqdn]

And restart Apache with service apache2 restart

BitTorrent Sync

If you’re not already familiar with BitTorrent Sync, it is a free, secure option for synchronizing directory structures in real time. I use it to synchronize the app between different web servers. Any changes are immediately propagated everywhere. This is also the vehicle for delivering updates to the server from the people who manage it.

This command will download the script to install version 1.4 of BitTorrent sync. Note that this is not the most recent version, as the new version is very limited in features and requires much more resources to run.
sh -c "$(curl -fsSL http://debian.yeasoft.net/add-btsync14-repository.sh)"

Then, run this command to install BitTorrent Sync
apt-get update && apt-get install btsync

Now we need to clean up any permissions and ownership issues with the following commands;

chmod 775 /var/www/ -R
chown www-data:www-data /var/www/ -R
chown www-data:btsync /var/www/f2.tech2u.com -R

Then we configure btsync to synchronize the app’s folder, and it does the work of importing the app.

Now I simply add the old BitTorrent key to the correct directory and then all the files copy over!

 

Eureka!

screenfetch

Moving My App To The Cloud

I have spent the better part of the last three years building a scalable logistics platform which has grown to manage nearly all the daily operations of my workplace, a mobile tech support company.

Some Major Features;

  • Booking appointments and scheduling them for employees in different regions and markets
    • Providing a portal for employees to see their jobs and communicate status to central dispatch
  • Handling accounting
    • Lots of custom reporting and alerts built in
  • Payroll calculations
    • Lots of different pay structures and commission rates
  • Reporting on sales data
    • Accounting and strategy development
    • Employee development
  • Outreach
    • Automatic outbound emails and calls for lots of different segmentations and purposes
    • Outbound sales for well-qualified customers
    • Follow-up surveys for quality assurance and customer satisfaction reporting
  • Business Intelligence
    • Predictive algorithms identify logistical problems before they happen and suggest fixes

This app started out as a way to make my job easier, managing daily operations and logistics. It quickly grew to take over much of our operations management, facilitating daily operations across the country and automating many previously tedious procedures and functions.

As it grew, it became more mission-critical.It currently provides essential infrastructure to many employee roles and daily operations.

A recent series of power outages proved that hosting it in-house was a bad idea, so I have begun to work on a solution: migration to the cloud.

Let’s Get Started

  1. The first step was to build a VPS using my Setting Up WordPress on a VPS tutorial to build the VPS, and then instead of installing WordPress, I copied over my app’s PHP files from its previous server.
  2. Next, I needed a secure connection to the existing databases until they can be migrated as well.
    I considered using a VPN, but this would be unduly complex because of the way our corporate network is setup.SSH is a great alternative to VPN.

    I used SSH to create a secure tunnel from the office to the new VPS. SSH has a feature which allows you to bind a port on the local machine to a port on the remote machine. In this way, I was able to connect from the VPS to the SQL server in the office.

  3. Next, I needed to rebuild many aspects of my app to use locally cached data instead of remote data. I built a new tool which continuously synchronizes a local copy of the remote database. This means that the remote database connection can fail without affecting the app. This parallels the future use-case when many companies may use the app, and all their data would need to be synchronized into the local database as well.

Migrating our Enterprise Production Environment From ESXi to Xen

When I started developing software at my current workplace, our web app server was running Turnkey Linux on Debian 6 inside a hypervisor running VMware ESXi 4.5; all of these tools were already very obsolete when I joined the development environment, and after over two years, it was time to make some changes.

debianI decided to move to a modern, open source hypervisor and the current version of Debian without the hundreds of irrelevant packages that come with Turnkey Linux.

I started by creating a new VM and installing Debian 8 with Apache, PHP and MySQL and then I migrated all the PHP scripts and the database over from the old web app VM to this new virtual server.

Now it was time to create a temporary hypervisor while I upgrade the current physical server. I found a disused workstation from the office and installed XenServer on it.

Both VMWare and Xen have a feature where you navigate to the server’s IP address in a browser and they give you a download for the desktop management tool to let you work with the server. Installing these was simple enough, then I exported all the old ESXi hypervisor’s virtual machines to OVF files, a universal standard format for virtual machine migrations.

Importing them into XenServer took A LONG TIME; I ended up leaving it overnight. But there were no problems on any of the VMs; Windows or several flavors of Linux, everything went very smoothly.

With the new VMs successfully imported, I shut down the old VMs and Started them new ones up. They took over their same static IPs and booted up as though nothing had happened.

virtualization-missingNow it was time to upgrade the old VMWare ESXi server to Xen. This process was a lot harder than I expected, and I did encounter two problems trying to get it to boot up for the first time.

Because it was an older Dual-Xeon server, it was missing an architecture feature required for some types of virtualization. It said Windows VMs might potentially have issues. This is fine for me because I am not using any Windows VMs. I looked at Dell’s website and there are no BIOS updates currently available which appear to resolve this issue. It may be impractical to use an older Dell server like this for hosting windows machines with Xen.

panicAnother problem arose because the older hyperthreading architecture threw a non-maskable interrupt parity error which caused a Kernel Panic and halted the machine. It took hours of research and work to solve this problem.

The first step was to disable hyperthreading in bios. Dell couldn’t make this simple; my server’s bios referred to it as disabling the “Logical Processor Feature.” Now the server was able to boot up, though it was only using one core on just one of its CPUs, instead of all the cores on both Xeons.

In order to prevent this Kernel panic, we need to tell the system to skip parity checks for NIM (or non-maskable interrupts). This is theoretically simple enough, but finding the bootloader configuration file proved very difficult as it was not in any of the normal places.

Once in the “Console” section of the XenCenter tool, the next step was finding the bootloader configuration file. This may differ for different versions of Xen, mine is 6.5. I eventually found the file here;

 /boot/extlinux.conf 

Find the section that looks like this…

label xe
  # XenServer
  kernel mboot.c32
  append /boot/xen.gz dom0_mem=752M,max:752M watchdog 

We need to add “nmi=ignore” into the “append” section like so;

label xe
  # XenServer
  kernel mboot.c32
  append /boot/xen.gz dom0_mem=752M,max:752M watchdog nim=ignore

After saving this file, I was able to reboot and turn the “Logical Processor Feature” back on, enabling all the cores and CPUs in the server. Then I was rewarded with a happy boot screen;

booting-up

Moving the virtual machines back to this server was as simple as moving them to the temporary one. Now everything is setup and running happily on this new, modern and open source hypervisor! 😀