Author: Robert Starsi

Script to update PhpBB 3.0.x to 3.1.x

Posted on by Robert Starsi

Recently I had to upgrade a dozen PhpBB boards to the latest version. Previously I would do this by hand, which would take days. This time though, while reading through the update notes, I noticed that it is possible to update the database through the console (I assume this was only introduced in 3.1). That was the necessary prerequisity to be able to automate everything. I ended up with the following script:

#!/usr/bin/env bash
echo "Upgrading PhpBB instal in '$1' to 3.1 with files from '$2'..."

cd $1

echo "Deleting old files..."
shopt -s extglob
rm -r !(config.php|images|files|store)
shopt -u extglob

echo "Copying in 3.1 files..."
rsync -a --exclude='.git' $2/ $1/

echo "Migrating the database..."
php ./bin/phpbbcli.php db:migrate --safe-mode

echo "Removing the install dir..."
rm -r install

echo "Done."

The usage is as follows (assuming you named the script phpbb-update-to-3.1):

phpbb-update-to-3.1 /www/forum-to-update /tmp/phpbb-3.1-files-for-upgrade

The second parameter is the path to where you have prepared your 3.1 files, according to this guide. You should put your custom styles in this folder too.

If you have custom folders or files within your current install, make sure you add those to the rm -r !(config.php|images|files|store) line in the update script. This rm deletes everything except the files within the curly braces.

This saved me a lot of time and prevented mistakes. Enjoy!

Bash: Underscore a String

Posted on by Robert Starsi

I find it weird that I did not find a ready made script for this immediately through a google search. So this is what I arrived at after some digging:

#!/usr/bin/env bash
read input
str="$(echo $input | tr '[:upper:]' '[:lower:]')"
printf ${str// /_}

Use it with stdin:

echo "Underscore Me" | underscore
underscore_me

I use this in TextMate when I need to convert normal english string/text into a parameter/key.

Automate everything

Posted on by Robert Starsi

To automate everything, even the smallest things (scripts), is a lesson I continuously learn while doing programming.

Let’s take MySQL and setting up replication for example. For years I would rely on googling and then copy and pasting the same few SQL commands to configure the master, slave and start the replication. Forward to around now, while crafting the Percona Docker Container I finally decided to encapsulate the process into 2 simple to use and remember commands.

Now, I can just exec into my MySQL container and type:

replication_master_sql

To get the SQL to execute on the master, tailored for the particular configuration. Once I execute this SQL on the master, I finish the whole “process” and start the replication with something like:

start_replication mysql-bin.000001 107

Where the last two arguments are the binlog position of where to start the replication.

When I first used this in production, the experience was like night and day compared to looking up the correct queries and then modifying them according to my setup. Thus—automate everything. It pays of every time you need it afterwards.

MailCatcher – My favorite development SMTP server

Posted on by Robert Starsi

Big part of development of webapps is to be able to effectively send and receive email from these apps. Using and external SMTP server is slow: you have to wait for the email to be send and then receive it through a real email client. Having a local SMTP server on the dev machine is a much better solution. I use MailCatcher. It’s open source, simple and today I fixed the only thing that was bothering me on it:

Latest iteration of klevo.sk

Posted on by Robert Starsi

To start 2015 in style, I’ve got a new theme for this website. Build from ground up to put the focus on my portfolio, clear typography and content. CSS were hand crafted, mobile first. On the backend – good old WordPress. Since all the cool internet kids are switching to https – I am following suit. For extra speed (and a free SSL certificate) I am utilizing CloudFlare to sit in front of the host.

All and all, I am satisfied and I hope all this new shiny tech will motivate me to write some quality content this year. Stay tuned!

Homepage serves to inform the visitor about what this website is about, put the most important content forward as well as create a good first impression.
Homepage serves to inform the visitor about what this website is about, put the most important content forward as well as create a good first impression.

Dockerized Percona MySQL Server with automated replication, tools & tests

Posted on by Robert Starsi

Docker & Percona Server I developed this container to solve specific needs and alleviate pain points present when dealing with deployment and administration of MySQL on servers that I manage.

I decided to look into Docker during a migration from MySQL 5.5 to 5.6 on one of the production servers. The server hosts multiple applications and services and is running in a hot spare configuration (another server mirrors this server, acts as the MySQL slave, etc.). Thus I wanted a migration strategy where I can have the 5.6 installation ready and running on the server, so that I can test it with production data and when ready just turn the switch to replace the old 5.5. Docker turned out to be the cleanest solution.

Since the MySQL server is such a critical part of the infrastructure I decided to develop the container utilizing test driven development. This allowed me to quickly add new features, like performance optimization and replication over a ssh tunnel (to support servers in different public clouds). Having this functionality in a standalone, tested and isolated unit is amazing. Before, all this complexity would be managed by Chef provisioning, which is much harder to test and experiment with on the production server. Having this functionality contained in a Docker container allows me to just use Chef for orchestration and deployment of the containers themselves, witch requires much simpler logic, compared to provisioning a full MySQL server install, configuration, replication and upgrading.

  • Code is available at Github.

This project is released under the MIT license.

DevOps

Posted on by Robert Starsi

serversSoon after I started working with Rails, I wanted to start using my own server environment, instead of a managed hosting. I wanted to be able to use any kind of libraries, their versions and any other tools my projects required on my server, set up the way I need it. Thus I got my first VPS.

From one small VPS server, over a period of a few years, I find myself managing 6+ dedicated servers to serve the needs of the applications I’ve developed and maintain.

At the beginning I managed my first server manually: ssh into the machine, install packages, edit the configs… This soon became tedious and I was aware that there are tools designed to do this better. I picked up Chef and never looked back. Nowadays all my servers are managed by cookbooks, deployed by chef-solo in conjunction with my own knife-like toolset based on Mina.

Here’s a short list of some of the more interesting things that I’ve learned and have experience with in conjunction with DevOps:

  • MySQL configuration, backup & replication (I’ve developed my own Docker container for this).
  • Using Postfix as a relay for services like Mandrill for reliable transactional email delivery.
  • Nginx, Unicorn, Upstart for reliably running apps with hot reloads on deploys.
  • Infrastructure monitoring using Zabbix (I’ve got my own container for that too).
  • Recovery from a failed disk in software RAID environment.
  • Setting up rate limiting and basic DDOS attack precautions.
  • Defending against small attacks like spam bots, password guessers, etc.

All of these skills translate and support my software development and application design, as I’m much more aware of the whole lifecycle of an application, and the real world usage scenarios that will be thrown at it.

EQAFE – Ecommerce

Posted on by Robert Starsi

This is one of the first projects where I started using Rails in production. EQAFE developed—over the years—from a simple checkout page to sell a single book to an ecommerce platform featuring thousands of products, affiliate program, gifts and more.

Deliverables:

  • UI design and implementation.
  • Backend design and implementation (Rails).
  • Automated test suite.
  • ~4 years running in production (DevOps).

Tour

Front page provides an index of the latest products.
Front page provides an index of the latest products.
Product details.
Product details.
A lot of the functionality is neatly tucked away in the always accessible dropdown menus.
A lot of the functionality is neatly tucked away in the always accessible dropdown menus.

Rails stats

eqafe-code-stats
It’s interesting to observe how each project can have quite a different allocation of the amount of code. Here for example, test vise, most of the code is located in controller tests – this tells us a lot about the app itself – a lot of the control flow is located in the controllers, most of the interesting stuff happens there, like for example users are redirected to different locations depending on their starting point and actions.

DIP Lite: E-learning Platform

Posted on by Robert Starsi

DIP Lite is a free e-learning platform. Its fully automated in-browser experience connects the users with a personally assigned “buddy”: an experienced person that overviews and assist the user throughout the course. The user walks the course on her own pace as well as engage in daily exercises, complete with email notifications and communication with the course buddy provided by the course platform.

For this project I delivered the following:

  • Initial design of the flow and functionality of the platform based on the goals set by the client.
  • Mockups of the UI and user flows.
  • Graphic design.
  • Logo creation.
  • Implementation within Rails.
  • Automated test suite.
  • Further feature development and updating/upkeep of the codebase.
  • Running this project in production for 3 years (DevOps).

Tour

Front page.
Front page.
After the lightning fast registration users immediately start with Lesson 1.
After the lightning fast registration users immediately start with Lesson 1.
Reviewing user's feedback.
Reviewing user’s feedback.
Admin dashboard.
Admin dashboard.

Rails stats

Code stats.
Code stats.

Projects Overview in 2012

Posted on by Robert Starsi

These are some of the projects I’ve launched during 2012. This year was interesting as I switched from mainly using PHP as a backend language to Ruby (on Rails). Switching to Ruby was a welcome refreshment, the syntax is so close to natural language. I really enjoy the way it allows one to express the idea in code. There is little I need to say about Rails.

Here are the projects I’ve been busy with this year:

Equal Money System website redesign

Desteni I Process redesign

EQAFE – Self-Perfection Merchadise