Troubleshooting my Raspberry Pi's Wireless Issue

It has been almost a week since I decided to self-host my Ghost blog. It was a fun experience and most importantly, I knew a lot of new things that I would not otherwise know. On the less technical side, it inspired me to write more about my learning journey because not only does it solidify what I already know, it also drives me to learn more.

There is a little problem though. My Internet connection is flaky and it causes my blog to be sporadically down throughout the day. This is not intended to be a for-profit blog, however, seeing people share some of my posts while my blog is down was frustrating. I just had to do something about it. I observed the Pi's behavior by writing several BASH scripts and cron jobs that makes sure these events are logged. Sifting through the logs after work, I found out that aside from the ISP problem, there is another queer phenomenon that was happening. Whenever my home router loses Internet connection, the Raspberry Pi will lose its default gateway; it persists even after rebooting the router.

My initial attempts to fix this issue was to mess with the resolve.conf and /etc/network/interfaces configuration files. I tried everything from manualdhcpand even static. Nothing really fixed the issue and it was still losing the default gateway route whenever the Internet connection goes down. I finally solved this problem by writing a small BASH script:

#!/bin/bash

ping -c1 google.com > /dev/null

if [ $? != 0 ]
then
  echo `date` "No network connection, restarting wlan0" >> /home/uplogs.txt
  /sbin/ifdown 'wlan0'
  sleep 5
  /sbin/ifup --force 'wlan0'
else
  echo `date` "Internet seems to be up" >> /home/uplogs.txt
fi 

The script pings google.com and then checks the exit code. If the ping exited with an error, the Pi restarts the wireless LAN interface. It also logs all these events so that I can check how reliable my Internet connection was throughout the day. It was a quick and dirty fix. Nothing fancy, but it works.

Getting started with tmux

I have been using tmux for several years now and it has since become a central part of my workflow as a software developer. Since I am constantly writing code, executing shell commands or accessing server instances via SSH, most of these things are done in the terminal. I am always on the lookout for cool and new tools that could potentially improve my workflow, so I checked tmux out. I knew I just have to get some hands on experience with it to find out just where it fits in my current flow.

tmux is being described as a terminal multiplexer. When I was first starting out, it was such a big word that added appeal to it. I thought it was leet, especially when I was still new to it. During my first days, I was using it solely for the sake of using it.

At this time, it became so ingrained into my system that the first thing I do upon arriving at work is spawn a terminal window into full screen and set the tmux windows that I will be using throughout the day.

Before I start with the basic commands though, I have to clear up a preconception that some people have about it. No, it does not manage your SSH connections. I need to stress this out because I have a certain colleague in the past who told me it was an ancient tool and dismissed it as a trend among hipster developers. He said he's better off using PAC Manager for all his SSH needs. These tools are apples and oranges. They complement each other; I also use PAC Manager because there is no way I will remember all the usernames and the host addresses I need to work with throughout the day.

To give a simple description as to what tmux is, you have to think of it as a serverthat serves terminal sessions. That allows you to attach and detach from it at will, and also gives other people a chance to attach to your existing tmux session. That is the main feature that makes it so awesome for everyone who works with remote stuff. Let us say that you have a VPS instance somewhere and you need to do some maintenance work. You SSH into your server, tell tmux to create a new terminal session and proceed with your work. After fifteen minutes or so, you remember that you have an important meeting to attend. The problem is that you aren't quite yet done with your work. As a contrived example, perhaps the server is doing a vulnerability scan or building something from source. Since you are attached to a tmux session, you can just kill your SSH connection. In tmux terms, it is referred to as detaching. After the meeting, you can attach back to your session and you will be presented with exactly the same screen as when you left. This enables you to see the scan results or the build progress without digging into the logs or trying to remember how it was doing before you left.

Another benefit of using tmux is, you will use your mouse less often once you get the hang of it. If you spend most of the day coding, reaching for the mouse to switch files or scroll through your code breaks the cadence. These are small personal idiosyncrasies, however, if you are plagued by the same quirk, you might want to learn VIM as well.

The good thing is that you only need to know a few commands to use tmux effectively. There are a whole lot of features and customization options available but you can learn them along the way. If you have used Emacs before, these commands will make you feel at home as the key combinations are somehow similar.

Outside a tmux session

Creating a new session

tmux new -s [session name]

Listing sessions

tmux ls

Attaching to an existing session

tmux attach -t [session name]

Inside a tmux session

Splitting the screen vertically

Ctrl - b % 

Splitting the screen horizontally

Ctrl - b "

Pane Navigation

Ctrl - arrow keys

Maximize a pane (from splitting)

Ctrl - b z

Closing a pane (from splitting)

Ctrl - d

Opening a new window

Ctrl - b c

Renaming a window

Ctrl - b ,

Window Navigation

Ctrl - b n

or

Ctrl -b p

Closing a window

Ctrl - b &

Detaching from a session

Ctrl - b d

I hope I have covered enough of the basics to get you started. Happy hacking!


Weekend Project: Self-hosted blog & Docker in a Raspberry Pi

I received a Raspberry Pi 3 Model B last Christmas, but I did not know what to do with. Or at least not yet. The problem has little to do with the Pi and more of the fact that most of the projects that I do can easily be solved with an Arduino.

When I stumbled upon these series of posts by the Docker Captain Alex Ellis, I figured out that this is a perfect opportunity to learn a tool I have always wanted to use. I know virtual machines well, but I had a hard time understanding how to make Docker fit into my workflow. The idea of containers that I cannot simply SSH into (I now know that you can exec bash to peek inside them, but that's not the point), just seemed absurd when I was first trying to use it. To be honest it felt too complex and cumbersome that I just dismissed it as something that was not worth it. Well, it turned out that I did not understand the philosophy behind it. I would like to talk about it and discuss images and containers in depth, but I decided that it will be better to have a dedicated post for that. After getting my hands dirty with Docker last weekend, I can say that I have attained a working proficiency with it and I can comfortably use it for my projects from here on.

After three days, I finally got it to work. The blog that you are reading right now is hosted on a Raspberry Pi with Docker Engine installed. I have two Docker containers running: the Ghost blog and the NGINX server that handles the caching. It took me a lot of trial and errors before I finally got it to work; I do not have any prior knowledge of NGINX when I embarked on this weekend project. The Pi's limited hardware made building images painstakingly slow. Building SQLite3 from source for the ARM architecture was excruciating.

I will be sharing my Dockerfiles and some configuration below. I won't go into more detail right now, but I am hoping that I will have the time to do so in my next post. Some of these are directly forked/copied from [Alex]((http://blog.alexellis.io)'s GitHub repositories; I could have pulled the images from Docker Hub or cloned the Dockerfiles but I decided to train my muscle memory by typing the Dockerfiles manually. I still have a lot to learn about NGINX and Docker in particular, but I consider this blog as a milestone.

Ghost Dockerfile

FROM alexellis2/node4.x-arm:latest

USER root
WORKDIR /var/www/
RUN mkdir -p ghost
RUN apt-get update && \
    apt-get -qy install wget unzip && \
    wget https://github.com/TryGhost/Ghost/releases/download/0.11.4/Ghost-0.11.4.zip && \
    unzip Ghost-*.zip -d ghost && \
    apt-get -y remove wget unzip && \
    rm -rf /var/lib/apt/lists/*

RUN useradd ghost -m -G www-data -s /bin/bash
RUN chown ghost:www-data .
RUN chown ghost:www-data ghost
RUN chown ghost:www-data -R ghost/*
RUN npm install -g pm2

USER ghost
WORKDIR /var/www/ghost
RUN /bin/bash -c "time (npm install sqlite3)"
RUN npm install

EXPOSE 2368
EXPOSE 2369
RUN ls && pwd

ENV NODE_ENV production

RUN sed -e s/127.0.0.1/0.0.0.0/g ./config.example.js > ./config.js
CMD ["pm2", "start", "index.js", "--name", "blog", "--no-daemon"]

Blog Dockerfile

FROM johncrisostomo/ghost-on-docker-arm:0.11.4

ADD Vapor /var/www/ghost/content/themes/Vapor

RUN sed -i s/my-ghost-blog.com/blog.johncrisostomo.com/g config.js

NGINX Dockerfile

FROM resin/rpi-raspbian:latest

RUN apt-get update && apt-get install -qy nginx

WORKDIR /etc/nginx/

RUN rm /var/www/html/index.nginx-debian.html && \
    rm sites-available/default && \
    rm sites-enabled/default && \
    rm nginx.conf

COPY nginx.conf /etc/nginx/

COPY johncrisostomo.com.conf conf.d/

EXPOSE 80

CMD ["nginx", "-g", "daemon off;"]

JOHNCRISOSTOMO.COM.CONF

server {
  listen 80;
  server_name blog.johncrisostomo.com;
  access_log /var/log/nginx/blog.access.log;
  error_log /var/log/nginx/blog.error.log;

  location / {
    proxy_cache              blog_cache;
    add_header X-Proxy-Cache $upstream_cache_status;
    proxy_ignore_headers     Cache-Control;
    proxy_cache_valid any    10m;
    proxy_cache_use_stale    error timeout http_500 http_502 http_503 http_504;

    proxy_set_header  X-Real-IP $remote_addr;
    proxy_set_header  Host      $http_host;
    proxy_pass        http://blog:2368;
  }
}

docker-compose.yml

version: "2.0"
services:
  nginx:
    ports:
      - "80:80"
    build: "./nginx/"
    restart: always

  blog:
    ports:
      - "2368:2368"
    build: "./blog.johncrisostomo.com/"
    volumes:
      - ghost_apps:/var/www/ghost/content/apps
      - ghost_data:/var/www/ghost/content/data
      - ghost_images:/var/www/ghost/content/images
      - ghost_themes:/var/www/ghost/content/themes
    restart: always

volumes:
   ghost_apps:
      driver: local
   ghost_data:
      driver: local
   ghost_images:
      driver: local
   ghost_themes:
      driver: local

I have written several follow up posts about this project. Feel free to check them out as most of them are troubleshooting issues and optimizations that are built on top of this project.


Cebu Mechanical Keyboard Enthusiasts Meetup 8/6


I have been busy these past few weeks since I started working on a server monitoring application. It will be released later this year; I have been busy and didn't have much time to blog and share about the new things that I have learned.

Despite the busy schedule, I managed to attend my first Mechanical Keyboard meetup at the Coffee Factory. Here are some of the awesome keyboards at the event.


Using Gagarin’s DDP Client to test Meteor methods, publications and subscriptions

On my previous post, we briefly went over unit testing in Meteor and Mantra by using Sinon’s spy and stub. We have discussed the difference between the two functions and determined when to use them for our unit tests.

Today, we are going to go through basic integration testing with methods and publications/subscriptions using Gagarin’s DDP client. Gagarin is the Mantra spec’s recommended testing framework for doing integration testing. It is versatile and can do a lot more that what we are going to cover here, like using chromedriver and Selenium for end to end testing, for example.

About Gagarin

According to the project’s documentation,

Gagarin is a mocha based testing framework designed to be used with Meteor. It can spawn multiple instances of your meteor application and run the tests by executing commands on both server and client in realtime. In other words, Gagarin allows you to automate everything you could achieve manually with meteor shell, browser console and a lot of free time. There’s no magic. It just works.

Gagarin is based on Laika, another testing framework created by Arunoda. According to the documentation, it can be thought of as Laika 2.0, though it is not backward compatible. The main differences between Gagarin, Laika and Velocity can also be found on the documentation above.

Installation

We can simply install Gagarin by running

npm install -g gagarin

Once we have written some tests, we can just run this command at the root of our app directory:

gagarin

By default, it will look for files that are in the tests/gagarin/ directory. It will build our app inside .gagarin/local/ along with the database that it will use for the duration of the test, which can be found at .gagarin/db/.
Now that we have a basic understanding of how to install and run Gagarin, let’s proceed and look at the code snippets that we are going to test.

Meteor code snippets (for testing)

In order for us to understand what we are testing, the code snippets will be included here so we can easily reference the functions and how they are being tested. I have simplified these code snippets so that we can focus more on testing.

Collection

const Categories = new Mongo.Collection(‘categories’);

Publications

Meteor.publish(‘categoriesList’, () => {
  return Categories.find()
});
Meteor.publish(‘categoriesOwnedBy’, (owner) => {
  check(owner, String);
  
  return Categories.find({owner: owner});
});

Methods

Meteor.methods({
  ‘categoriesAdd’(data) {
     check(data, Object);
 
     Categories.insert({
       name : data.name,
       owner: data.owner,
       createdAt : new Date(),
     });
   },
   ‘categoriesUpdate’(data) {
     check(data, Object);
     
     Categories.update({_id:data._id},{$set:{
       name : data.name, 
       owner: data.owner,
       modifiedAt : new Date(),
     }});
   },
});

Writing Gagarin Tests

Now that we have seen the code that we are going to test, we can now start writing basic tests in a single JavaScript file inside tests/gagarin/. Because Gagarin is based on Mocha, it has the same describe — it structure. Chai’s expect is also exposed for doing more semantic assertions.

Testing the categoriesAdd method

The test we are going to do first is to check whether or not we can add something to the categories collection.

describe(‘Categories’, function() {
  var app = meteor({flavor: “fiber”});
  var client = ddp(app, {flavor: “fiber”});
  it(‘should be able to add’, function() {
    client.call(‘categoriesAdd’, [{name: ‘First category’}]);
    client.sleep(200);
    client.subscribe(‘categoriesList’);
    var categories = client.collection(“categories”);
    expect(Object.keys(categories).length).to.equal(1);
 }); 
}

We are defining the initial describe block that we are going to use for this example. Gagarin gives us two useful global functions that are essential for running tests: meteor and ddp.

meteor is used to spawn a new Meteor instance that we have assigned to the app variable. Meteor uses fibers by default, so we need to specify it as the flavor. ddp allows a client to connect to the Meteor instance that we have just created by passing the reference of the instance and the flavor as its arguments.

Since we now have our Meteor app and our client configured, we are ready to proceed with our first test case: making sure that we can successfully add a new category.

Inside our it block, we are calling the Meteor method categoriesAdd. Gagarin provides our client with a handy call function that works exactly the same way as Meteor.call. The only difference is that the arguments need to be inside an array, regardless of its number.

We then use the sleep function to add a little delay so that we can make sure that the new document comes to the client. We are subscribing to our categoriesList publication through the handy subscribe function of our client. Just like the call function, this is similar to Meteor.subscribe, which makes it very straightforward.

After subscribing to our publication, we now check if the document has been inserted by our Meteor method to the collection. We do that by calling the collection function of our client, passing the name of the MongoDB collection as an argument. It returns an object that looks like this:

{ Hpu6Z4h7ZFtC6Q77m: 
 { _id: ‘Hpu6Z4h7ZFtC6Q77m’,
 name: ‘First category’,
 owner: null,
 modifiedAt: 2016–06–07T08:29:06.026Z,
 createdAt: 2016–06–07T08:29:06.026Z } }

It looks similar to something that we would get if we query our collection using find, aside from the fact that instead of getting back an array or a cursor, we are getting an object which has the _id field as a key.

We then use Chai’s expect function to do a simple assertion and that completes our first test. Object.keys has been used on the object that was returned by the collection function, so we can just expect the resulting array to have a length of 1. This test makes us sure that the client can call our method, and can receive the the document through our publication.

Testing the categoriesUpdate method

We now have a basic test that checks if we can insert and retrieve documents, what we want to do next is to check if we can update a certain category from our collection. The process is similar to what we did on the last section (still goes inside the same describe block):

it(‘should be able to update’, function() {
  client.subscribe(‘categoriesList’);
  var categories = client.collection(“categories”);
  var id = Object.keys(categories)[0];
  client.call(‘categoriesUpdate’, [{_id:id, name: ‘updated    category’}]);
 client.sleep(200);
 
  categories = client.collection(“categories”);
  expect(categories[id].name).to.equal(‘updated category’);
});

The only thing that is new here is that we are storing the id of the category that we want to update so we can use it when we call categoriesUpdate. We can then check if the name has been updated by using expect.

Testing categoriesOwnedBy publication

The next thing that we will test is the categoriesOwnedBy publication. Since we did not use the owner field in our previous examples, we will put this test on a separate describe block. That will allow us to spawn a new Meteor and database instances that has nothing to do with the previous one.

describe(‘categoriesOwnedBy publication’, function() {
  var app = meteor({flavor: “fiber”});
  var client = ddp(app, {flavor: “fiber”});
  it(‘should only publish a specific users category’, function() {
    app.execute(function() {
      var categories = [
        { name: ‘Category 1’, owner: ‘John’ },
        { name: ‘Category 2’, owner: ‘Jessica’ },
        { name: ‘Category 3’, owner: ‘John’ }
      ];
      Meteor.call(‘categoriesAdd’, categories[0]);
      Meteor.call(‘categoriesAdd’, categories[1]);
      Meteor.call(‘categoriesAdd’, categories[2]);
   });
  client.subscribe(‘categoriesOwnedBy’, [‘John’]);
  
  var johnsCategories = client.collection(‘categories’);
  
  expect(Object.keys(johnsCategories).length).to.equal(2);
});

This looks similar to our two previous examples, but this time I am using the execute function of our Meteor instance. It accepts a callback function as an argument, and the contents of that function will be executed in the server context. Notice how we have access to Meteor.call inside this function?

We then go back to the client context and subscribe to our categoriesOwnedBypublication, passing ‘John’ as our argument. After fetching the contents of the collection, we are checking if we are getting the expected number of documents that was published by our collection.

Running our test

If we run gagarin on the root folder of our application, we will get something similar to this:

Conclusion

Using the examples above, we have seen how to create simple integration tests using Gagarin on Meteor. These test cases might seem contrived, but the idea here is to get an overview of how to use Gagarin’s DDP client to perform basic integration tests that deals with Meteor methods, publications and subscriptions.