Backup your Bluehost server remotely with Ansible

tl;dr I created a simple Ansible playbook to dump databases and folders from a Bluehost shared hosting, ideal for automating backups using a cron job, you can get the source code for this playbook here.

After all the incidents with the Heartbleed bug and security issues, I  decided to audit my servers to spot security flaws, some days ago I started redirecting all the system and application logs to papertrailapp.com a simple service to centralise all your logs and create alerts, search, store them, etc. After that I started thinking in a better way to secure my information and automate backups of multiple sites and databases I have a across multiple hosting providers.

One of those providers is Bluehost, a very popular shared hosting service. If you have some experience with shared hosting you will know there are things that simply are not possible to do in contrast with a VPS or dedicated server. Nevertheless there is a lot you can do if your provider let you access via ssh. In this post I’ll be showing you how to automate backups using Ansible a simple but powerful tool for managing and automate infrastructure. If you don’t have or dont know how to access Bluehost via ssh, read this guide.

The Bash way

The first thing that came to my mind before Ansible was a bash script a simple script that would be triggered by a cron job in my home server and use mysql-dump via SCP to dump and copy databases backups into my home server and  then rsync to maintain files and folders synced.

The method was fairly simple, but after being working with Ansible for some months I really wanted to create something simpler and cleaner, so after translating that bash script into Ansible  I ended up with this playbook:

If you are curious about the bash script, here is it (it doesn’t have the rsync part for folders, just mysql dump): https://gist.github.com/jcalonso/120f12270535e146226b

Finally if you want to schedule your backups, you can easily trigger this playbook from a cron job like this (put this in the server that will store the backups):

00 1 * * 0    cd /path/to/playbook; ansible-playbook -i hosts bluehostBackup.yml --tags databases folders

Here you can find the full repository: https://github.com/jcalonso/ansible-bluehost-backup

Let me know what you think in the comments below, and if you have a better ideas to improve it.

 

Your development environment with Vagrant and Ansible

Introduction

My development workflow has been always evolving. As a software developer I’m always trying to find the best combination of tools and configuration to be more efficient and reduce the number of steps of getting code written in my computer  into production. I remember those days when I preferred to write code and test it directly in production. My knowledge about how things worked in those years was so limited that I didn’t want to deal with any unknown error.

After some time I realised how wrong this was and started using tools like XAMPP/WAMP/MAMP to recreate a LAMP production environment in my machine. The applications were simple, no obscure configurations or dependencies were needed. This way of coding in local and shipping to production after testing locally seemed to be the right way to go, at least for a while. After joining Clear Books and started working with our application framework, things went more complicated. A simple LAMP stack wasn’t enough to get the whole application up and running, a lot of custom configuration were needed. If you were a linux user, you can just get a copy of a preconfigured virtual machine and start developing in a virtualised environment, on the other hand, if you wanted a high responsive native installation, you will need to spend around a day and the help from another developer to get it ready. This approach was wrong.

The Enlightenment

Right on time when I was having these issues and thinking of how to speed up the development process from getting a fresh clean OS installation to a fully development machine, I stepped into this project from GitHub called Boxen. Boxen pretends to automate all the tedious tasks of installing all the tools and dependencies for you. When Will Farrington was presenting Boxen at PuppetConf 2012, I literally jaw dropped after hearing how this piece of software was able to automate all the things for GitHub, and hopefully for you too. It is a great project but for now it works only on OS X, so I had to discard this at the moment.

Some time later I read a post on the Tuenti corporate blog about their full development workflow, and I was surprised to know that they were facing the same situation as me. The time to get a new developer ready to work was long, then they moved to local development servers per team, which was good but not enough, they needed more isolation. In their actual workflow now each developer can have their own environment  called “Tuenti-in-a-Box” which uses Vagrant for creating, managing and distributing portable environments using virtualisation and Puppet, a configuration management tool widely used by DevOps professionals.

The Vagrant and Ansible solution

After doing more research on Vagrant and Ansible, a configuration management tool which I first heard from another developer at Clear Books, I decided to use both as my combination for setting up a fast and improved development environment.

Some of the characteristics I wanted to achieve with this new workflow were:

  • Repeatable: By having all the setup and configuration task together with the source code, you are pretty sure that every time you run them it will be the same result.
  • Verifiable: Since this configuration is written by people who knows the system internals, it makes this approach verifiable.
  • Isolated: The entire environment is isolated in a virtual machine.
  • Easy: Vagrant up: easier? impossible.
  • Fast: Since there are no processes where human needs to interact, the speed will be delimited by your hardware and internet connection.

Vagrant

Vagrant provides easy to configure, reproducible, and portable work environments built on top of industry-standard technology and controlled by a single consistent workflow to help maximize the productivity and flexibility of you and your team [1].

Basically what vagrant will enable you to do is creating configuration files (Vagrant file) with a set of rules that will define how a virtual machine should be built. A simple example of a Vagrant file is:

Ansible

Ansible is an IT automation tool. It can configure systems, deploy software, and orchestrate more advanced IT tasks such as continuous deployments or zero downtime rolling updates.

Ansible’s goals are foremost those of simplicity and maximum ease of use. It also has a strong focus on security and reliability, featuring a minimum of moving parts, usage of Open SSH for transport (with an accelerated socket mode and pull modes as alternatives), and a language that is designed around auditability by humans – even those not familiar with the program.[2]

For the use case that I’m talking in this post, Ansible will contain all the specifications of the tools and configurations we needed to install for the web application. This configuration is hold on files called Ansible playbooks. This is a example of piece of a Ansible playbook for setting up a web server with php:

One of the differences that made me chose Ansible from Puppet was the simplicity of setup. Ansible doesn’t require a puppet agent in the guest machine which was a great plus. The other characteristic was the language used to write the configuration files (ansible playbooks), while puppet uses a custom json syntax, Ansible chose yaml with a good set of best naming practices to manage all the instructions to run in the guest machine.

Conclusion

The days of hand-crafting your development environment have ended, the complexity of new information systems is growing at big scale and solutions like Vagrant and Ansible are just one of the options to achieve a configurable, reproducible and portable environment, tools like puppet, chef, salt stack should be considered too.

Configuration and infrastructure should be treated as code in order to avoid time consuming tasks and lower error rates. Development environments should imitate the production as much as possible, and your testing and staging environment should do the same.

References:
[1] Vagrant. (Nov 12, 2013). Vagrant Docs:
Retrieved from:
http://docs.vagrantup.com/v2/why-vagrant/index.html

[2] Ansible Works. (Nov 12, 2013). Ansible Works Documentation:
Retrieved from:
http://http://www.ansibleworks.com/docs/

Time Management

[vía KXCD]

Easy way to upload files from a *nix server to a Synology NAS

Yesterday finally arrived the new NAS (Synology RS409) for the SEL Lab At UC3M. The first task was to configure and make a mirror of all the data on our old file server.

There are several ways to do this by command line, for example, SCP with SSH or rsync. At least this 2 options were the first that came to my mind but they didn’t work nice.

First, SCP seems to be the best way to do this, but what a surprise!, the Synology Software DSM 3.0 have a “light” version of SSH that doesn’t support SCP.

So the next move was rsync since it was supported I thought it will be a another good way, but the problem now was the speed, rsync spends a lot of CPU usage on the checksum verification (the NAS have a small ARM processor). So instead of having speeds about 25 – 30 mb/s I got an average of 5 mb/s.

I did a little research and I found that the easiest way to do this (Without having to install anything via ssh on the NAS) was using lftp this way:

#install lftp on the old server

sudo apt-get install lftp

#connect to the server

lftp -d [remoteServer] – u [username]

#make a reverse mirroring

mirror -R [localFolder] [remoteFolderToPutTheFiles]

And that’s all.

Enjoy!

 

iBooks se actualiza a la 1.2

La aplicación de iBooks para iPhone e iPad se actualizan dando al fin la posibilidad de ordenar nuestros libros y pdfs en “colecciones”.
Entre otras características:

* ligero cambio en la GUI
* posibilidad de imprimir mediante air print

iPhone 4

Pues después de varios meses buscandolo al fin me pude hacer de un iPhone 4 que compré en la Apple store de Park Meadows en Denver CO, lugar en el que estuve tomando una capacitación para desarrollar aplicaciones de iOS, pero eso es otro post (espero).

Pues por ahora tengo un bonito iPod touch ya que para activarlo es necesario esperar a que saquen el unlock para iPhone 4 con IOS 4.2.1 con baseband 03.10.01.

Ya que tenga mas noticias actualizaré este post (espero)

Saludos!

Leganés – Casa de Campo – Tour por el centro de Madrid

Mi primer recorrido con un integrante del equipo de MTB de LeganésRuteriño

Muy buen recorrido, tranquilo para ir retomando el nivel, aparte de que fue muy educativa al pasar por los principales lugares históricos de Madrid.


Cambio de dominio

Que tal!!

Hace ya varios meses que escribí mi ultimo post y me siento mal por ello, es por eso que nuevamente regreso con nuevas ideas para que el posting sea mas continuo. Intentare hacer reviews de Apps para iPhone/iPod touch. De mi opinión sobre noticias relevantes en el mundo de la informática, subir fotos. comentar eventos y cosas que sucedan en mi vida.

Como pueden ver, he migrado mi blog a otro dominio y en su propio subdominio que esto me dejará hacer más cosas bajo la dirección jcalonso.com y no solo tener un blog

Espero empezar a tene un poco mas de lectores y cualquier cosa dejen sus comentarios y yo los responderé!

Saludos

P.D. Use este post también para probar la app de wordpress para iPhone

Juan Carlos Alonso

Baratijas Podcast 05: El geek vagabundo

Nuevamente los invito a escuchar este podcast sobre tecnologia en el que estoy involucrandome cada vez mas.

Temas

* Tecnologia: Google Music, Navigation y Street View en 7 ciudades de México.

* Software Libre: Ubuntu 9.10, Cherokee Web Sever

* Eventos: Campus Party: conferencias, talleres, etc.

* Gadgets: Motorola DEXT datos oficiales para México, Motorola DROID, Sony Ericsson Xperia X10 (Rachael).

* Esta semana en Apple: Las nuevas Macs, problemas en la iMac de 27 pulgadas, escacez de magic mouse y mac mini server.
* Apple “bloquea” instalar Snow Leopard 10.6.2 en un Intel Atom.

Baratijas Podcast 05: El geek vagabundo

Unboxing del Garmin Forerunner 310xt

Un rápido unboxing de este fantástico reloh deportivo de Garmin. Disculpen el nivel del volumen tan bajo.

Espero poder escribir un review mas completo proximamente.