Getting started with tmux

I have been using tmux for several years now and it has since become a central part of my workflow as a software developer. Since I am constantly writing code, executing shell commands or accessing server instances via SSH, most of these things are done in the terminal. I am always on the lookout for cool and new tools that could potentially improve my workflow, so I checked tmux out. I knew I just have to get some hands on experience with it to find out just where it fits in my current flow.

tmux is being described as a terminal multiplexer. When I was first starting out, it was such a big word that added appeal to it. I thought it was leet, especially when I was still new to it. During my first days, I was using it solely for the sake of using it.

At this time, it became so ingrained into my system that the first thing I do upon arriving at work is spawn a terminal window into full screen and set the tmux windows that I will be using throughout the day.

Before I start with the basic commands though, I have to clear up a preconception that some people have about it. No, it does not manage your SSH connections. I need to stress this out because I have a certain colleague in the past who told me it was an ancient tool and dismissed it as a trend among hipster developers. He said he's better off using PAC Manager for all his SSH needs. These tools are apples and oranges. They complement each other; I also use PAC Manager because there is no way I will remember all the usernames and the host addresses I need to work with throughout the day.

To give a simple description as to what tmux is, you have to think of it as a serverthat serves terminal sessions. That allows you to attach and detach from it at will, and also gives other people a chance to attach to your existing tmux session. That is the main feature that makes it so awesome for everyone who works with remote stuff. Let us say that you have a VPS instance somewhere and you need to do some maintenance work. You SSH into your server, tell tmux to create a new terminal session and proceed with your work. After fifteen minutes or so, you remember that you have an important meeting to attend. The problem is that you aren't quite yet done with your work. As a contrived example, perhaps the server is doing a vulnerability scan or building something from source. Since you are attached to a tmux session, you can just kill your SSH connection. In tmux terms, it is referred to as detaching. After the meeting, you can attach back to your session and you will be presented with exactly the same screen as when you left. This enables you to see the scan results or the build progress without digging into the logs or trying to remember how it was doing before you left.

Another benefit of using tmux is, you will use your mouse less often once you get the hang of it. If you spend most of the day coding, reaching for the mouse to switch files or scroll through your code breaks the cadence. These are small personal idiosyncrasies, however, if you are plagued by the same quirk, you might want to learn VIM as well.

The good thing is that you only need to know a few commands to use tmux effectively. There are a whole lot of features and customization options available but you can learn them along the way. If you have used Emacs before, these commands will make you feel at home as the key combinations are somehow similar.

Outside a tmux session

Creating a new session

tmux new -s [session name]

Listing sessions

tmux ls

Attaching to an existing session

tmux attach -t [session name]

Inside a tmux session

Splitting the screen vertically

Ctrl - b % 

Splitting the screen horizontally

Ctrl - b "

Pane Navigation

Ctrl - arrow keys

Maximize a pane (from splitting)

Ctrl - b z

Closing a pane (from splitting)

Ctrl - d

Opening a new window

Ctrl - b c

Renaming a window

Ctrl - b ,

Window Navigation

Ctrl - b n

or

Ctrl -b p

Closing a window

Ctrl - b &

Detaching from a session

Ctrl - b d

I hope I have covered enough of the basics to get you started. Happy hacking!


Weekend Project: Self-hosted blog & Docker in a Raspberry Pi

I received a Raspberry Pi 3 Model B last Christmas, but I did not know what to do with. Or at least not yet. The problem has little to do with the Pi and more of the fact that most of the projects that I do can easily be solved with an Arduino.

When I stumbled upon these series of posts by the Docker Captain Alex Ellis, I figured out that this is a perfect opportunity to learn a tool I have always wanted to use. I know virtual machines well, but I had a hard time understanding how to make Docker fit into my workflow. The idea of containers that I cannot simply SSH into (I now know that you can exec bash to peek inside them, but that's not the point), just seemed absurd when I was first trying to use it. To be honest it felt too complex and cumbersome that I just dismissed it as something that was not worth it. Well, it turned out that I did not understand the philosophy behind it. I would like to talk about it and discuss images and containers in depth, but I decided that it will be better to have a dedicated post for that. After getting my hands dirty with Docker last weekend, I can say that I have attained a working proficiency with it and I can comfortably use it for my projects from here on.

After three days, I finally got it to work. The blog that you are reading right now is hosted on a Raspberry Pi with Docker Engine installed. I have two Docker containers running: the Ghost blog and the NGINX server that handles the caching. It took me a lot of trial and errors before I finally got it to work; I do not have any prior knowledge of NGINX when I embarked on this weekend project. The Pi's limited hardware made building images painstakingly slow. Building SQLite3 from source for the ARM architecture was excruciating.

I will be sharing my Dockerfiles and some configuration below. I won't go into more detail right now, but I am hoping that I will have the time to do so in my next post. Some of these are directly forked/copied from [Alex]((http://blog.alexellis.io)'s GitHub repositories; I could have pulled the images from Docker Hub or cloned the Dockerfiles but I decided to train my muscle memory by typing the Dockerfiles manually. I still have a lot to learn about NGINX and Docker in particular, but I consider this blog as a milestone.

Ghost Dockerfile

FROM alexellis2/node4.x-arm:latest

USER root
WORKDIR /var/www/
RUN mkdir -p ghost
RUN apt-get update && \
    apt-get -qy install wget unzip && \
    wget https://github.com/TryGhost/Ghost/releases/download/0.11.4/Ghost-0.11.4.zip && \
    unzip Ghost-*.zip -d ghost && \
    apt-get -y remove wget unzip && \
    rm -rf /var/lib/apt/lists/*

RUN useradd ghost -m -G www-data -s /bin/bash
RUN chown ghost:www-data .
RUN chown ghost:www-data ghost
RUN chown ghost:www-data -R ghost/*
RUN npm install -g pm2

USER ghost
WORKDIR /var/www/ghost
RUN /bin/bash -c "time (npm install sqlite3)"
RUN npm install

EXPOSE 2368
EXPOSE 2369
RUN ls && pwd

ENV NODE_ENV production

RUN sed -e s/127.0.0.1/0.0.0.0/g ./config.example.js > ./config.js
CMD ["pm2", "start", "index.js", "--name", "blog", "--no-daemon"]

Blog Dockerfile

FROM johncrisostomo/ghost-on-docker-arm:0.11.4

ADD Vapor /var/www/ghost/content/themes/Vapor

RUN sed -i s/my-ghost-blog.com/blog.johncrisostomo.com/g config.js

NGINX Dockerfile

FROM resin/rpi-raspbian:latest

RUN apt-get update && apt-get install -qy nginx

WORKDIR /etc/nginx/

RUN rm /var/www/html/index.nginx-debian.html && \
    rm sites-available/default && \
    rm sites-enabled/default && \
    rm nginx.conf

COPY nginx.conf /etc/nginx/

COPY johncrisostomo.com.conf conf.d/

EXPOSE 80

CMD ["nginx", "-g", "daemon off;"]

JOHNCRISOSTOMO.COM.CONF

server {
  listen 80;
  server_name blog.johncrisostomo.com;
  access_log /var/log/nginx/blog.access.log;
  error_log /var/log/nginx/blog.error.log;

  location / {
    proxy_cache              blog_cache;
    add_header X-Proxy-Cache $upstream_cache_status;
    proxy_ignore_headers     Cache-Control;
    proxy_cache_valid any    10m;
    proxy_cache_use_stale    error timeout http_500 http_502 http_503 http_504;

    proxy_set_header  X-Real-IP $remote_addr;
    proxy_set_header  Host      $http_host;
    proxy_pass        http://blog:2368;
  }
}

docker-compose.yml

version: "2.0"
services:
  nginx:
    ports:
      - "80:80"
    build: "./nginx/"
    restart: always

  blog:
    ports:
      - "2368:2368"
    build: "./blog.johncrisostomo.com/"
    volumes:
      - ghost_apps:/var/www/ghost/content/apps
      - ghost_data:/var/www/ghost/content/data
      - ghost_images:/var/www/ghost/content/images
      - ghost_themes:/var/www/ghost/content/themes
    restart: always

volumes:
   ghost_apps:
      driver: local
   ghost_data:
      driver: local
   ghost_images:
      driver: local
   ghost_themes:
      driver: local

I have written several follow up posts about this project. Feel free to check them out as most of them are troubleshooting issues and optimizations that are built on top of this project.


Cebu Mechanical Keyboard Enthusiasts Meetup 8/6


I have been busy these past few weeks since I started working on a server monitoring application. It will be released later this year; I have been busy and didn't have much time to blog and share about the new things that I have learned.

Despite the busy schedule, I managed to attend my first Mechanical Keyboard meetup at the Coffee Factory. Here are some of the awesome keyboards at the event.


Using Gagarin’s DDP Client to test Meteor methods, publications and subscriptions

On my previous post, we briefly went over unit testing in Meteor and Mantra by using Sinon’s spy and stub. We have discussed the difference between the two functions and determined when to use them for our unit tests.

Today, we are going to go through basic integration testing with methods and publications/subscriptions using Gagarin’s DDP client. Gagarin is the Mantra spec’s recommended testing framework for doing integration testing. It is versatile and can do a lot more that what we are going to cover here, like using chromedriver and Selenium for end to end testing, for example.

About Gagarin

According to the project’s documentation,

Gagarin is a mocha based testing framework designed to be used with Meteor. It can spawn multiple instances of your meteor application and run the tests by executing commands on both server and client in realtime. In other words, Gagarin allows you to automate everything you could achieve manually with meteor shell, browser console and a lot of free time. There’s no magic. It just works.

Gagarin is based on Laika, another testing framework created by Arunoda. According to the documentation, it can be thought of as Laika 2.0, though it is not backward compatible. The main differences between Gagarin, Laika and Velocity can also be found on the documentation above.

Installation

We can simply install Gagarin by running

npm install -g gagarin

Once we have written some tests, we can just run this command at the root of our app directory:

gagarin

By default, it will look for files that are in the tests/gagarin/ directory. It will build our app inside .gagarin/local/ along with the database that it will use for the duration of the test, which can be found at .gagarin/db/.
Now that we have a basic understanding of how to install and run Gagarin, let’s proceed and look at the code snippets that we are going to test.

Meteor code snippets (for testing)

In order for us to understand what we are testing, the code snippets will be included here so we can easily reference the functions and how they are being tested. I have simplified these code snippets so that we can focus more on testing.

Collection

const Categories = new Mongo.Collection(‘categories’);

Publications

Meteor.publish(‘categoriesList’, () => {
  return Categories.find()
});
Meteor.publish(‘categoriesOwnedBy’, (owner) => {
  check(owner, String);
  
  return Categories.find({owner: owner});
});

Methods

Meteor.methods({
  ‘categoriesAdd’(data) {
     check(data, Object);
 
     Categories.insert({
       name : data.name,
       owner: data.owner,
       createdAt : new Date(),
     });
   },
   ‘categoriesUpdate’(data) {
     check(data, Object);
     
     Categories.update({_id:data._id},{$set:{
       name : data.name, 
       owner: data.owner,
       modifiedAt : new Date(),
     }});
   },
});

Writing Gagarin Tests

Now that we have seen the code that we are going to test, we can now start writing basic tests in a single JavaScript file inside tests/gagarin/. Because Gagarin is based on Mocha, it has the same describe — it structure. Chai’s expect is also exposed for doing more semantic assertions.

Testing the categoriesAdd method

The test we are going to do first is to check whether or not we can add something to the categories collection.

describe(‘Categories’, function() {
  var app = meteor({flavor: “fiber”});
  var client = ddp(app, {flavor: “fiber”});
  it(‘should be able to add’, function() {
    client.call(‘categoriesAdd’, [{name: ‘First category’}]);
    client.sleep(200);
    client.subscribe(‘categoriesList’);
    var categories = client.collection(“categories”);
    expect(Object.keys(categories).length).to.equal(1);
 }); 
}

We are defining the initial describe block that we are going to use for this example. Gagarin gives us two useful global functions that are essential for running tests: meteor and ddp.

meteor is used to spawn a new Meteor instance that we have assigned to the app variable. Meteor uses fibers by default, so we need to specify it as the flavor. ddp allows a client to connect to the Meteor instance that we have just created by passing the reference of the instance and the flavor as its arguments.

Since we now have our Meteor app and our client configured, we are ready to proceed with our first test case: making sure that we can successfully add a new category.

Inside our it block, we are calling the Meteor method categoriesAdd. Gagarin provides our client with a handy call function that works exactly the same way as Meteor.call. The only difference is that the arguments need to be inside an array, regardless of its number.

We then use the sleep function to add a little delay so that we can make sure that the new document comes to the client. We are subscribing to our categoriesList publication through the handy subscribe function of our client. Just like the call function, this is similar to Meteor.subscribe, which makes it very straightforward.

After subscribing to our publication, we now check if the document has been inserted by our Meteor method to the collection. We do that by calling the collection function of our client, passing the name of the MongoDB collection as an argument. It returns an object that looks like this:

{ Hpu6Z4h7ZFtC6Q77m: 
 { _id: ‘Hpu6Z4h7ZFtC6Q77m’,
 name: ‘First category’,
 owner: null,
 modifiedAt: 2016–06–07T08:29:06.026Z,
 createdAt: 2016–06–07T08:29:06.026Z } }

It looks similar to something that we would get if we query our collection using find, aside from the fact that instead of getting back an array or a cursor, we are getting an object which has the _id field as a key.

We then use Chai’s expect function to do a simple assertion and that completes our first test. Object.keys has been used on the object that was returned by the collection function, so we can just expect the resulting array to have a length of 1. This test makes us sure that the client can call our method, and can receive the the document through our publication.

Testing the categoriesUpdate method

We now have a basic test that checks if we can insert and retrieve documents, what we want to do next is to check if we can update a certain category from our collection. The process is similar to what we did on the last section (still goes inside the same describe block):

it(‘should be able to update’, function() {
  client.subscribe(‘categoriesList’);
  var categories = client.collection(“categories”);
  var id = Object.keys(categories)[0];
  client.call(‘categoriesUpdate’, [{_id:id, name: ‘updated    category’}]);
 client.sleep(200);
 
  categories = client.collection(“categories”);
  expect(categories[id].name).to.equal(‘updated category’);
});

The only thing that is new here is that we are storing the id of the category that we want to update so we can use it when we call categoriesUpdate. We can then check if the name has been updated by using expect.

Testing categoriesOwnedBy publication

The next thing that we will test is the categoriesOwnedBy publication. Since we did not use the owner field in our previous examples, we will put this test on a separate describe block. That will allow us to spawn a new Meteor and database instances that has nothing to do with the previous one.

describe(‘categoriesOwnedBy publication’, function() {
  var app = meteor({flavor: “fiber”});
  var client = ddp(app, {flavor: “fiber”});
  it(‘should only publish a specific users category’, function() {
    app.execute(function() {
      var categories = [
        { name: ‘Category 1’, owner: ‘John’ },
        { name: ‘Category 2’, owner: ‘Jessica’ },
        { name: ‘Category 3’, owner: ‘John’ }
      ];
      Meteor.call(‘categoriesAdd’, categories[0]);
      Meteor.call(‘categoriesAdd’, categories[1]);
      Meteor.call(‘categoriesAdd’, categories[2]);
   });
  client.subscribe(‘categoriesOwnedBy’, [‘John’]);
  
  var johnsCategories = client.collection(‘categories’);
  
  expect(Object.keys(johnsCategories).length).to.equal(2);
});

This looks similar to our two previous examples, but this time I am using the execute function of our Meteor instance. It accepts a callback function as an argument, and the contents of that function will be executed in the server context. Notice how we have access to Meteor.call inside this function?

We then go back to the client context and subscribe to our categoriesOwnedBypublication, passing ‘John’ as our argument. After fetching the contents of the collection, we are checking if we are getting the expected number of documents that was published by our collection.

Running our test

If we run gagarin on the root folder of our application, we will get something similar to this:

Conclusion

Using the examples above, we have seen how to create simple integration tests using Gagarin on Meteor. These test cases might seem contrived, but the idea here is to get an overview of how to use Gagarin’s DDP client to perform basic integration tests that deals with Meteor methods, publications and subscriptions.


Using Sinon’s Spy and Stub in Mantra (Unit Testing)

With the release of Meteor 1.3, unit testing has never been easier in Meteor. Our team recently decided to adopt Arunoda’s Mantra spec for developing Meteor applications. It is an application architecture which allows for a more modular approach with clear separation of concerns with the client and the server side. It has been a rough month for us since the spec is new and there are limited learning resources available. There were also a lot of things to learn since adopting the Mantra spec meant that we have to learn React for the presentation logic, instead of sticking with Blaze.


Unit testing is something that can be easily accomplished with the Mantra spec. Since it is modular and it clearly separates the presentation logic from the business logic through containers, components and actions, everything can be unit tested. Meteor 1.3 also introduced native NPM support, which means that using familiar tools such as Mocha, Chai and Sinon can be imported in a straightforward manner. This is going to be my first blog post and so I am only going to explain Sinon’s spy and stub methods as they were used in Arunoda’s mantra-sample-blog application.

Spies

People who are new to unit testing tools in JavaScript (with me included) were initially confused about the difference between Sinon’s spy and stub. It turns out that a spy is a basic function that you can use in Sinon, and that stubs and mocks were built on top of it.

According to the Sinon.JS documentation, a spy is

a function that records arguments, return value, the value of this and exception thrown (if any) for all its calls. A test spy can be an anonymous function or it can wrap an existing function

What this means is that a spy can be used as a replacement for an anonymous callback function or you can wrap an existing function into it so that you can spy on its behavior. For example, if you have a function that accepts another function as an argument to be called back later after a certain condition, you can instead pass Sinon’s spy() function as the callback. You can then assert or use Chai’s expect() to see if that function was called by using spy()’s calledOnce(). Additionally, you can check if the correct arguments were passed to it by using its calledWith(). You can check the different spy functions that are available here.

Now let’s look at how spies are used in the Mantra sample blog application. We are going to use the tests that were written for the posts action. Let’s check this code snippet out:

it('should call Meteor.call to save the post', () => {
  const Meteor = {uuid: () => 'id', call: spy()};
  const LocalState = {set: spy()};
  const FlowRouter = {go: spy()};
  
  actions.create({LocalState, Meteor, FlowRouter}, 't', 'c');
  
  const methodArgs = Meteor.call.args[0];
  
  expect(methodArgs.slice(0, 4)).to.deep.equal([
    'posts.create', 'id', 't', 'c'
  ]);
  expect(methodArgs[4]).to.be.a('function');
});

In the test block above, we are testing if our action will correctly invoke a Meteor Method in the server through Meteor.call(). The first three lines are creating local objects for MeteorLocalState and FlowRouter to be used exclusively in this test case. In the Mantra spec, these are exported as the application context inside client/configs/context.js.

Actions in Mantra receive this application context as the first parameter. We are creating local objects in lieu of the real app context and by doing so, we can trick the action into thinking that it is receiving its first expected argument (which is the app context).

Next, we spy on how these objects are used inside the action that we are testing. See how the Meteor object that we have passed contains a call property, which is a Sinon’s spy() function? When the action gets invoked on Line 6, it will go ahead and invoke Meteor.call() inside it. The Meteor object that it will receive is something that we have just created for spying purposes, so we have access to the arguments that were passed into it when it was invoked (Lines 7 — 12). We used the arguments that we have obtained through spying to verify that our function is invoking Meteor.call() with the correct arguments.

This is what the create action looks like, for reference:

  create({Meteor, LocalState, FlowRouter}, title, content) {
    if (!title || !content) {
      return LocalState.set('SAVING_ERROR', 'Title & Content are required!');
    }

    LocalState.set('SAVING_ERROR', null);

    const id = Meteor.uuid();
    Meteor.call('posts.create', id, title, content, (err) => {
      if (err) {
        return LocalState.set('SAVING_ERROR', err.message);
      }
    });
    FlowRouter.go(`/post/${id}`);
  }

Stubs

Now that we are done with spies, and have a basic understanding on how they work, let’s work with stubs. Stubs are just like spies, and in fact they have the entire spy() API inside them, but they can do more than just observe a function’s behavior. According to the Sinon API, stubs are:

functions (spies) with pre-programmed behavior. They support the full test spy API in addition to methods which can be used to alter the stub’s behavior.

and they should be used when you want to:

Control a method’s behavior from a test to force the code down a specific path. Examples include forcing a method to throw an error in order to test error handling.

or

When you want to prevent a specific method from being called directly (possibly because it triggers undesired behavior, such as a XMLHttpRequest or similar).

Okay, so where spies can observe and spy on how a function is going to be called, the number of times it’s going to be called and the arguments that were sent with it, stubs can do all these, plus you can also programmatically control its behavior.

Let’s check how it is used on the post action test inside the sample blog application:


    it('should call Meteor.call to save the post', () => {
      const Meteor = {uuid: () => 'id', call: spy()};
      const LocalState = {set: spy()};
      const FlowRouter = {go: spy()};

      actions.create({LocalState, Meteor, FlowRouter}, 't', 'c');
      const methodArgs = Meteor.call.args[0];

      expect(methodArgs.slice(0, 4)).to.deep.equal([
        'posts.create', 'id', 't', 'c'
      ]);
      expect(methodArgs[4]).to.be.a('function');
    });

This particular test will check if our action will set an error message if something goes wrong after the Meteor Method call. Just like what we did with the spy example, we are setting local objects to be used as the context for our action function.

In Mantra, the LocalState is a Meteor reactive-dict data structure (a reactive dictionary) which is mainly used to handle the client side state of the app, although it is mostly used to store temporary error messages. We are creating a LocalState object here to mimic the app context’s LocalState. We are setting its set property as a spy function, so we can later see if our action will set the appropriate error message by checking the arguments that were passed into it.

Notice that this time, we are using a stub() instead of a spy() for our local Meteor object. The reason for this is that we are no longer just observing how it is going to be called, but we are also forcing it to respond in a specific way.

We are checking our action’s behavior once the call to a remote Meteor method returns an error and if that error will be stored in the LocalState accordingly. In order to do that, we need to reproduce that behavior, or make the call() function in our local Meteor object return an error. That is something that a spy() will not be able to do since it can only observe. For this scenario, we will use the stub’s callsArgWith()* function to set our desired behavior (Line 6).

We will give callsArgWith() two arguments: 4 and the err object that we have defined in Line 5. This function will make our stub invoke the argument at index 4, passing the err as an argument to whatever function is at that position. If you are going to look at our create action above, Meteor.call() is invoked with five arguments, the last or the one in the fourth index is a callback function:

Meteor.call(‘posts.create’, id, title, content, (err) => { 
  if (err) { 
    return LocalState.set(‘SAVING_ERROR’, err.message); 
  } 
});

We have to remember that this Meteor.call() that is being invoked here is the local object that we have created and passed explicitly into our create action for testing purposes. As such, this is the stub in action, and it doesn’t know that the last argument when it is invoked is going to be a callback function, so we have to use the callsArgWith() with the err object. Inside this callback, the create action will then store the error message in the LocalState object that we have passed in. Since the set() function of that LocalState object is a spy, we can conclude our test by checking if the arguments that were passed to this spy function matches the error message that we are expecting (Line 9).

This wraps up our discussion of how Sinon’s Spy and Stubs methods are being used in Mantra Unit Testing. As a recap, a spy just observes a certain function, or can take the place of a anonymous callback function so we can observe its behavior. A stub does more than that, by allowing us to pre-program a function’s behavior. If I have provided any wrong information, please feel free to correct me in the comments. :)

*The callsArg and yields family of methods have been removed as of Sinon 1.8. They were replaced with the onCall API.