Justin McCandless

Reading Molecular Data in JavaScript with PDB

Sun, 25 Feb 2018 21:05:50 -0800

The RCSB has a really great library of data for thousands of proteins in its Protein Data Bank, but sadly the formats that they offer are ancient. Both PDB and mmCIF formats use character indices to describe data fields, making them more suited to be read into FORTRAN than transferred over the modern web. After running into this problem enough times myself, I’m releasing the open source parse-pdb package in an attemp to save other people some time wrangling while loops and substring calls.

Getting Started

You can install the library just like you would any other npm module:

npm install --save parse-pdb

From there you’ll just import the module and call it with a string containing an entire PDB file. Node’s fs modules is nice for reading this info in from a file, but if you want to get it directly from RCSB or any other way it works too.

const parsePdb = require('parse-pdb');
const { readFileSync } = require('fs');

const pdbString = readFileSync('./3aid.pdb', 'utf8');

const parsed = parsePdb(pdbString);
// parsed now contains a nice javascript object with all the 3AID PDB data!

A Returned Format That’s Actually Useable

The object that you get back, called parsed in the example above, contains the basic data from the PDB more or less as-is, but it also has some more user-friendly derived data as well.

The basic data is parsed from the ATOM and SEQRES entries in the PDB. These are both arrays of objects that contain every data value in the row as a key/value pair. See the output format documentation for a list of every property and its type.

console.log(parsed.atoms); // [ { serial: 1, ... }, ...1845 others ]


console.log(parsed.seqRes); // [ { serNum: 1, ... }, ...1845 others ]

In addition, I’ve also parsed out residues and chains for convenience and given them references to their lower level elements. For example, you can get residues in a chain or atoms in a residue:

const residue = parsed.residues[0];
const residuesAtoms = residue.atoms; // array of 9 objects from parsed.atoms

const chain = parsed.chains.get(residue.chainID);
const siblingResidues = chain.residues; // array of 99 residues

Let’s Make Protein Data Easier to Work with on the Modern Web

I hope this parser can be of use to other people and help bridge the gap between scientific computing and the modern web. However, I’m more comfortable with the computing side than the science side, and this is just my guess at what others might find useful. If you’ve got any ideas for what might make this more useful to you, open an issue (or a pull request!) on the Github repository, or just drop a comment below.

"A Review of Bluegogo Bike Share" on Medium

Mon, 20 Feb 2017 21:05:50 -0800

I just posted a review of the bike share service Bluegogo over on Medium, which you can check out right here.

As someone who's interested in transportation and especially bicycling, I had a lot of fun biking around town trying out the service, taking pictures, and writing about my experience. I've been longingly looking at the Bay Area Bike Share stations for awhile, sad that they don't make any financial sense for my use cases, so I was happy that something new has come to town.

Enjoy the article, and hopefully I'll have some new tech stuff for this blog shortly.

HTML5 Audio with React

Sat, 12 Mar 2016 22:48:42 -0800

The HTML5 audio tag provides a great way to work with simple audio in a web app, but when you’re using React’s nice pattern in the rest of your app, it can be a pain to go back to messing directly with DOM objects. The ReactAudioPlayer module provides a thin wrapper to the audio tag that fits right into a React app.

ReactAudioPlayer is something I wrote while working on an audio project. It passes through all of the typical attributes that the audio tag uses (listed nicely on the MDN audio article). Most importantly, it provides a nice interface to all of the events fired by the audio tag. So instead of messing with references to your audio tag and setting event listeners everywhere, you can do this:

<ReactAudioPlayer
  onPause={this.onPause}
/>

This follows for any other event you can listen to on the audio tag. And of course, any changes to the passed in props will update the audio tag immediately. This includes the src attribute, which will change the currently playing or loaded media in the audio tag.

ReactAudioPlayer is an npm module, so you can install it in your app with npm install --save react-audio-player. Check it out on npm and on github.

Setting up HTTPS on Node for Free with Let's Encrypt

Sat, 27 Feb 2016 22:08:27 -0800

Given how vitally important running your site in HTTPS is, it's disappointing how high the barrier to entry is. Some services like Heroku do make it easy to set up, but you will have to pay for that simplicity.

Recently, the EFF and several other organizations put together a project called Let's Encrypt, which hopes to lower this barrier to entry. Let's Encrypt is a free certificate authority and collection of software that makes it easy to obtain and autorenew certificates for any site. As someone who is passionate about the cause for encryption, but has also cheaped out on spending the time and money to set up HTTPS on a few too many side projects, this was exactly what my conscience needed.

Getting set up was not without its headaches though; Let's Encrypt is a new and very ambitious project with support spread across many different platforms. In this article, I'll do my best to point you in the right direction using what I learned while setting up HTTPS for an Express app.

Let's Encrypt Express

We're going to be using a great library called letsencrypt-express. This will generate our certificate, store it on the filesystem, and renew the certificate whenever needed, without taking down the server.

If you were to jump into Let's Encrypt straight from the getting started guide on their site, you'd realize how much letsencrypt-express is doing for you. The Let's Encrypt site introduces a command line tool for generating certificates that is much more manual. If you're looking to use Let's Encrypt on another platform or on a server like nginx, that might be a good place to start, but if you're on Node, stick with letsencrypt-express or other implementations by the same author, Daplie.

Code

The letsencrypt-express README has two different sections of code that you'll need to include in your app (if you just want to see it already set up in a working app, check out my example project). The first section is setup, which creates the lex object with some configuration. Be sure to change the email to your actual email address, and also take a look at configDir. This is where your certificates will be installed. Your app will have to have read and write permissions in this directory.

The Express section has the next bit of code you'll need to include. It sets up your app to listen on both production HTTP and HTTPS ports, so if your situation is different, be sure to change those ports. In reality you'll probably want to set up a conditional depending on what environment your app is running on.

Run It

If you run the server at this point, accessing it at a non-HTTPS port should access your app like normal. Accessing it via HTTPS should also bring up your app, and if you look at the server's output on the command line, you'll see detailed output telling you that letsencrypt-express has generated and started using your certificate. If you open the directory you specified earlier, you should see a folder containing the generated files.

Troubleshooting

With this setup, you'll have to be doing this on a web server; the app won't load via HTTPS on your local machine. If you try you might get an error like Error creating new authz :: DNS name does not have enough labels.

Also, as of this writing, the current version of letsencrypt-express has a few problems. If you get the error TypeError: dest.end is not a function when accessing the server, then you might want to try downgrading to the 1.1.1 version. If the server complains about missing the http2 module when starting, just install it manually.

If you can't access your app via HTTPS at all, make sure that your server is configured to allow connections on port 443.

Testing Mode

Assuming you got everything running using the code from the README, you are currently running in testing mode. letsencrypt-express uses a non-production certificate in this mode that your browser might complain about. To use it in production mode and use a certificate that your browser should accept, just remove the .testing() call and include it like this instead: var LEX = require('letsencrypt-express');.

From Here

When your certificate expires, letsencrypt-express should generate a new one in the background without you needing to do anything. You now have a working HTTPS app with a free and open certificate authority! Now go use this setup for all your apps and let's build a more secure internet.

Using ES6 Generators to Manage Dialog Flow in a Game

Sun, 19 Apr 2015 21:07:44 -0700

Remember how great it was when you first started using promises to combat javascript callback hell? Your code was much more concise, easier to understand, and didn't nest a million levels deep...

Well EcmaScript 6 generators promise (no pun intended) to bring about a similar revolution in the way we deal with asynchronous code. We've been hearing about how complex code dealing with API calls and user interactions will be able to look just like synchronous code, with everything happening in one flow and being much easier to follow. It sounded great, I looked over some brief code examples and it made sense, I played around with it myself and started to understand all the function*s and yields, but I was waiting for one more thing: a example of a real world app benefiting from generators.

I ran into a nice use case in a side project today, so I decided to write that generator example app myself. Feel free to jump ahead to the full live demo and its source code.

Game Dialog Good and Bad

A series of dialog text in a game is a great example of where generators excel. In the game designer's mind, a series of dialog blurbs come into the game one after the other in a very linear and simple way based on the user's input. Using callbacks and nesting each blurb another level deep just gets in the way.

makeTextbox('Hello!  Click to continue.');

events.on('click', function() {

  makeTextbox('I\'m the second text bubble, yay!');

  events.on('click', function() {
    makeTextbox('I hope we don\'t have too many more of these...');
  });
});

There is no benefit to the developer for nesting all of these like this. And this is not even considering branching options or more complex conditions.

With generators, we can flatten it out like this:

makeTextbox('Hello!  Click to continue.');

yield events.on('click', function() {
  makeTextbox('I\'m the second text bubble, yay!');
  generator.next();
});

yield events.on('click', function() {
  makeTextbox('And we can have as many more as we want without nesting.');
  generator.next();
});

How Generators Work

I won't bother repeating what's already well documented on MDN and explained in a very thorough article at 2ality, but here's the basics that you need to get through this article.

When using ES6 generators, you'll be declaring a generator function with the function* syntax:

function* gen() { 
  yield 1;
  yield 2;
  yield 3;
}

Then using it to create a generator:

var g = gen();

And calling next to progress through the yield statements:

g.next()  // 1
g.next()  // 2
g.next()  // 3

Notice how in the generator function, execution pauses until we call next again. This is what we will exploit to write code that "waits" for something external to happen, like an API call to return or a user to perform an action.

Getting Set Up

As of this writing, native support for generators is not very thorough. An easy way to get around this and start using generators now is to use something like Babel, a compiler that compiles ES6 to ES5.

Even easier, my example project mentioned above (available on Github) will give you a full setup with Gulp, Browserify, and Babel, and compile any changes to the code automatically for you.

A Fully Featured Example

Go ahead and run through the live demo really quickly. Let's take a look at the generator code that makes that happen (you can see the full source of this file here).

That sure is a lot easier to read than a ton of nested callbacks or promises.

We're using the yield operator to show when our code will pause, and this.flowGen.next() to indicate that the previous event has happened and we can continue. The result is a flat code structure despite having many layers of asynchronous code waiting for user input.

From Here

I encourage you to pull down the demo source code and play around with it or build something yourself, as building something real that takes advantage of generators is really the hardest part of learning them.

Getting a Distributed Static Website Running with Maelstrom

Sun, 12 Apr 2015 12:21:50 -0700

BitTorrent's Project Maelstrom is a web browser for the decentralized web that downloads assets via torrent instead of web server. It was recently released to the public as a beta, and with it a simple Python tool to create torrents optimized for Maelstrom from static websites. That got me really excited, so I went through the simple process of serving up my own static site via torrent.

What better content to serve this way than Multitasq, a backend-less HTML5 app I wrote awhile back. If you're following along with the article and need something to serve up, grab the source code, run npm install, bower install, and gulp, and then all your static assets will be in the dist/ directory.

Creating an Optimized Torrent

First we're going to create the torrent file that will be used the same way that a url is used on the everyday web. Start by cloning BitTorrent's torrent-web-tools project from Github. Use generator.py as described in the README to create a torrent for whatever static site you want to serve. Make sure that you have an index.html in the root directory.

Right now it's also important to note that when we seed this torrent file with our torrent client, it's going to look for the files in its default download directory. So, you might want to move the root directory of your static website content into ~/Downloads/ or wherever that is before generating the torrent file.

generator.py ~/Downloads/multitasq/

Seeding Your Torrent

Now you just need to seed the torrent you created so that Maelstrom has a place to download the site's assets from. The torrent-web-tools README suggests using only uTorrent or BitTorrent for this right now, so open up either of those and add your torrent file. As long as it can find your files in the download directory ok, you should see that it's seeding your content.

Open it in Maelstrom

From here we can open up the site just like a normal site in a web browser. Open Maelstrom and then open the torrent file that you created. It might be slow at first, but you should see your torrent program start uploading the content of your torrent and Maelstrom should load up the page.

From Here

Anyone with the torrent file you created (or magnet link) should be able to use your website in Maelstrom, all without a centralized server. For example, you can view the copy of Multitasq that I created while writing this article here: bittorrent://f5baf0471f0d40a9b1dbd1b5fde3d22f7800aef4/index.html (sorry if I'm not always seeding!).

This technology has a lot of potential and I'm excited to see where it goes from here. I guess that depends on the developers that are using it and what we create though, so get hacking!

SVG viewBox sandBox

Sat, 11 Apr 2015 12:40:28 -0700

Awhile back I wrote an article about SVG viewBox after being confused by it for so long. Just recently I went back and wrote a live updating sandbox app in React that allows you to quickly play around with different viewBox setups and see the result instantly.

Check out the live demo here and see if you can follow how different changes to viewBox and preserveAspectRatio affect the SVG.

The source code is also available on Github.

And of course if you're still thoroughly confused by viewBox after all of the above, definitely take a look at my article Making Sense of SVG viewBox's Madness.

Formatting Currency in Javascript

Sat, 13 Dec 2014 17:28:26 -0800

I just had more trouble than I should have trying to figure out how to properly format currency, so it's going up as a blog post. Intl, the new feature available in modern browsers for internationaliztion, is actually really great for localizing and formatting things like currency, but here's to hoping this saves someone time digging through the spec.

Here's the code you want for USD:

var nf = new Intl.NumberFormat(‘en-US’, {
  style: ‘currency’,
  currency: ‘USD’,
  minimumFractionDigits: 2,
  maximumFractionDigits: 2
});
nf.format(123456.789); // ‘$123,456.79’

The minimumFractionDigits and maximumFractionDigits options are what took me so long to find. When you format currency, at least for USD in English, you usually don’t want any more than 2 “cents” places displayed, or any less as well (it’s $3.50, not $3.5). These two options will ensure you always get 2 digits of cents.

If you'd like more info on Intl, NumberFormat, and what they can do, MDN explains this very thoroughly. I ran into this while using Formatjs, which has been great while doing fully featured internationalization with Handlebars. Lastly, keep in mind that Intl is shimmed if you need better browser support.

A Starting Point for Routed Meteor Apps

Sat, 13 Dec 2014 16:52:26 -0800

There are a lot of really cool demos out there for simple, single page Meteor apps. When you want to get a bit more practical and add a few different states with routes, Meteor still works great, but the initial work for getting this running can be a bit more strenuous. This post and my open source boilerplate project Meteorplate aim to help get you up and running in the shortest time possible.

Get Started Now

If you want to play around with this and see if it fits your app, ~~check out the live demo~~. If you want to jump into the code right now, clone the Meteorplate Github repo, run meteor, and start hacking on your app.

Meteorplate

Back in the early days of Meteor, I recommended using the Backbone router and views to solve this problem in another article. Since then, Meteor has come a long way, and the community around it has too. This time around, we're going to solve this problem in Meteor 1.0+ using native Meteor templates directly as views and the widely used Iron Router for routes.

With just a little boilerplate, we'll allow page structure to be rendered with a changing main view, permissions for pages based on user accounts, and even let Google Analytics track page changes. All this will still be reactive and take full advantage of Meteor's live real-time setup.

App Structure

Meteorplate is fittingly using HTML5 Boilerplate, along with the typical Meteor client/, public/, and server/ directories at the top level, so the folder structure should be familiar. Inside of client/, we've put all of the templates in the templates/ directory as plain html (such as this one for the homepage).

Inside client/js/, main.js kicks off the app and sets everything up for Meteor. The views/ director contains all definitions of template methods and events (such as the colors page view).

The last big piece is where we use Iron Router in routes.js. In here, we've just provided a light wrapper around Iron Router's usual functionality in order to provide the nice features mentioned above. To add a route, just define a new Iron Router route in the define method and call render() with the Meteor template that you want to render.

All of this will provide you a nice setup for a fully featured Meteor app. To get started, clone the Meteorplate repo and start building your app quickly on top of this solid foundation.

Update

Meteor has stopped their free hosting period, which has killed my live demo for this project. If you want to try it, I encourage you to clone the repository and run it yourself with the meteor command.

Announcing Mentorthing

Sun, 12 Oct 2014 18:56:12 -0700

Today marks the soft launch of a project I've been working on with some friends called Mentorthing. I work on a lot of side projects for fun and for a technical challenge, but this one I'm really hoping can make a difference for a few people.

The idea behind Mentorthing is to create an open platform where anyone can put themselves out there as a mentor, and anyone else can seek their help as a mentee. No vetting process or exclusivity whatsoever.

I don't need to tell you how valuable a good mentor can be in any sort of education, but the problem is access to mentors. The current avenues that students and learners have for finding a mentor are often exclusive and antiquated, like alumni or corporate mentorship programs.

Mentorthing hopes to be an inclusive platform, where knowledgeable people can declare to the world that they're willing to help, and anyone has the ability to strike up a conversation and see if they could benefit. It's a system that I really hope can help a few people achieve their goals.

We're a small community that's just starting out, so if this sounds interesting to you, I encourage you to sign up and check it out. No pressure, you're totally able to just put in your info and wait and see if anyone is interested in talking to you. It's a work in progress, so feel free to reach out to me personally with suggestions or other ways to get involved.

Sharing JavaScript Between the Front and Back End

Sun, 12 Oct 2014 18:05:27 -0700

There are a lot of advantages to running Nodejs, but one that's not taken advantage of as much as it could be is sharing code between the front and back ends. You're running the same language on both sides of the stack, so why rewrite code that's run on both?

You can get pretty fancy with this. If you're familiar with Meteor, you know how easy this platform makes it to forget the distinction between frontend and backend altogether. Even without a huge framework though, doing something like sharing data models on both sides is quite a nifty trick to prevent having to write things like validation twice.

One place that this sort of code sharing can quickly and easily make a difference is in config. You probably have a few constants that are needed by both the frontend and backend, and instead of defining them twice or fetching them from the server, you can simply share the config code on both sides.

Here's how that might look:

This code could be run by the frontend and the backend without any trouble. On the frontend, config would be assigned as a global variable, and on the backend, it would be exported as a node module. All in the same file, without repeating any config.

This is a super simple example, but if you have been repeating any code in your javascript front and back ends, I hope you see how easy it is to share code to both.

Write Your Commit Message First and Code Second

Sun, 03 Aug 2014 11:08:45 -0700

It hit me the other day at work. I've been using git wrong all along. The workflow for using version control shouldn't be to write a hacked together summary of all the code changes I just made every time I get the project in a sane state. It should be to write a contract with myself describing the state of the project after some incremental improvement, and then to write that improvement. How did I ever think otherwise?

Improving Productivity

I'm a big fan of always striving to improve my productivity as a programmer, and one of the biggest lessons learned over the years has been to work in short, manageable chunks. This tries to maximize the use of our natural ability to focus, and it helps avoid the huge time sink of a project with tons of different ongoing improvements that will take forever to get back into a working state. Despite my best efforts though, scope always tends to creep broader and broader as I work.

Writing a commit message first works wonders on reigning this in. Whenever I start to get sidetracked with tangential work, I can look at my prewritten commit message to see exactly how the project should work when I'm finished. Whenever I get distracted by a meeting or come back from lunch, a quick look at my commit message and see exactly what I was working on. It helps me to plan in advance, to stay on track, and to deliver what I said I would when I actually do commit.

TDD Lite

You might be drawing similarities between this method and test driven development. Test driven development follows an even more strict adherence to this plan first, code second pattern, and there's a reason why many teams have found a ton of success with it. Test driven development requires quite a bit of overhead work and buy-in from the team, however, and might not be something everyone can spend the time to commit to.

Commit message first development follows a similar pattern and starts to provide a lot of the same benefits, but is almost painless to implement. You'll be planning out what you want your code to do, keeping yourself on track along the way, and delivering something to meet your original criteria just like in TDD. The pattern is similar, but it's a lot easier to get going.

How to Write a Commit Message First

Writing your commit messages first can be as easy as storing a string in a command line variable. Plan out the state your project will be in after a manageable chunk of coding, and save a message that describes it like this:

msg="Profile images over 4mb are rejected"

Later on while coding, if you need a reminder of exactly what it is you should be doing, you can output your saved message like this:

echo $msg

Finally, when you're ready to commit, this is how you would commit in git with your pre-saved message:

git commit -am "$msg"

Pretty simple workflow with just about zero overhead to get started.

Being an efficient programmer is something of a holy grail that can never be fully achieved, but if you work at it there are always new ways to improve. Writing my commit messages first has been a great way to keep myself working efficiently, and even to write better commit messages while I'm at it. It might not be perfect for every coding style, but there's not much holding you back from seeing for yourself how it works for you.

Building a Requirejs Project to a Single, Reusable File

Mon, 10 Mar 2014 00:03:32 -0700

Requirejs is great for breaking up JavaScript projects into multiple files, but what happens when you want to use your Requirejs code in another project? Ideally, no one should have to individually import your modules, or even use Requirejs at all. Luckily this is all part of the plan for Requirejs projects, and with a bit of tweaking with the finicky r.js tool, you'll be able to build to a single file with no dependencies.

Setup

We'll be using two tools to do this: the command line r.js tool, and almond. Feel free to use Grunt or Gulp with their respective Requirejs plugins grunt-contrib-requirejs and gulp-module-requirejs instead of the command line tool itself, as the config is exactly the same, but in this article we'll use the command line tool for simplicity.

Start off by installing r.js via npm:

npm install -g requirejs

Also grab the almond.js file either from the repo or from bower:

bower install almond

The Command Line Tool

You can find the official documentation for compiling a Requirejs project into one file under the optimization section of the Requirejs site. That means we're talking exclusively about using r.js with the -o flag, which is used like this:

r.js -o

Your build configuration can be one of two things. It can be a path to your build file, or it can be your series of options directly on the command line. In this article we're going to use a separate build file for clarity, but first an important distinction:

Your main Requirejs file is not the same thing as your build file.

You've probably written a main file for Requirejs before that looks something like this:

The config section might look similar to some of the build parameters given on the Requirejs site under optimization, but this is not what's used directly by r.js -o; you need a separate build configuration.

Build Config

The Requirejs optimization page also gives a description of most of the parameters used in the build file, and there is also a super detailed example in the repo. Our basic build file for the sake of this article looks like this:

Here, r.js would be run from the root of the project (containing the app/ folder) like this:

r.js -o build.js

This build file references a Requirejs config file with mainConfigFile; again note that they are two different things. r.js will look in the given config file for any Requirejs config given with requirejs.config. Most importantly, it will grab your paths parameter from there and use those paths to find all of your modules. If you're not using a mainConfigFile parameter in your build file, you can also specify the paths here, but this approach of referencing the config file avoids needing to repeat information.

baseUrl determines the directory for the later include parameter and also for the paths given in the config file.

out gives the output file path relative to where r.js was run.

The optimize parameter here is just telling r.js to minify the code when it concatenates everything, which is nice for your build process but not necessary.

include points to an array of main Requirejs files to include, or just the single main file in our case. Other files specified by paths will still be included without repeating them here. Note that this main file is the same as our mainConfigFile. That's because the this file contains our config as well as the entry point for our app and is used independently for each.

The name parameter is pointing to almond, relative to our baseUrl, which will include almond in the build process.

Keep in mind that r.js is incredibly finicky about these paths and you must get everything perfectly right!

The Result

Running this setup will gather all files in your project, concatenate them, minify them, and output them into the specified output file in a way that can be used by external projects.

However, if you're not using almond, you'll notice that the user of your project still needs to include Requirejs!

Almond

Almond allows you to build to a file without depending on Requirejs. It's a bare bones AMD loader that replaces the minimal needed functionality of Requirejs and sticks itself into your final built file.

If you included almond in the name attribute in your build file as shown above, your output file will be ready to go.

From Here

Requirejs can add a lot of headaches with how finicky it can be, but the benefits of breaking up your JavaScript nicely with AMD are pretty great. Stick to a configuration similar to the above and you should be able to get your setup working smoothly. And don't forget that you can stick the parameters in your build file into the task options for Grunt and use this easily in your build process.

A Tutorial for Getting Started with Gulp

Sun, 23 Feb 2014 16:50:45 -0800

Awhile back I wrote a tutorial covering the basics of Grunt. Now that a new frontend build tool has entered the scene, I thought I'd take a look and see if I could write a simple guide for Gulp at the same level. This guide will take you from installation and getting set up through examples of real world build processes while laying out all of the tools out there to help you on the way.

Resources

This article aims to be thorough, but you'll also be referring to a few other resources while working with Gulp.

The Gulp Github repo is the main place to go for help. The README has a nice sample gulpfile, and the repo also contains Gulp's documentation. The getting started doc shows you how to install and create a blank gulpfile.

The Gulp website mainly just refers to the Github repo, but it does contain an invaluable plugin directory as well.

Why a Frontend Build Tool?

Sure you can just upload your raw development files to a web server and call it a day, but there are a lot of tools out there that can make a developer's life a lot easier, from CoffeeScript to jshint to Jasmine testing. If you're going to utilize even a very simple build process, a tool like Gulp can make things much more automated.

Gulp is a command line tool that you run at various points in your build process?—?or even continuously. If your build process previously consisted of compiling Sass and minifying your JavaScript, Gulp could handle both of those for you at once, and it's much easier to configure than a bunch of bash commands. Even more importantly, it's portable thanks to npm, so you can build on any machine you check out your repo to after a simple npm install.

Gulp and Grunt

The main difference you'll want to keep in mind between Grunt and Gulp is that Grunt is controlled through configuration and Gulp is controlled through code. You tell Grunt what to do through a json file, but you create your Gulp build process by writing JavaScript and invoking Gulp's API.

Besides that, you've got two pretty similar tools. Both have the goal of automating your frontend build process. Both are written mainly in Node and set up with a variety of plugins using npm. And both have a friendly root level file to tell them what to do (Gruntfile.js and gulpfile.js).

Installation

The documentation contained in the Gulp Github repo for getting up and running is actually quite effective, so check out the official getting started guide referenced above. You'll just be installing Gulp globally and then locally to your project using npm, so make sure you have Node installed. Just two simple commands here:

npm install -g gulp
npm install --save-dev gulp

Hello World

You can continue with the getting started guide to create a minimal gulpfile. Here's what my idea of a super basic one might look like:

You could now jump back to the command line and run this with the command gulp. As you can probably tell, the only thing this does is output 'Hello world. '

Let's take a look at what 's going on in this gulpfile. The first thing you might notice is that this is just JavaScript, it's not a json config file like with Grunt. It's using Node's require function to include gulp, then defining a task.

Tasks are the same concept from Grunt: you can define arbitrary tasks and run them from the command line with gulp . default is the task called when no parameter is given. In Gulp though, tasks are just functions, and you can place whatever code you want inside of them.

A Useful Gulpfile - Copying Files

Now to a gulpfile that does something useful. This is might be a setup you have for a simple Yeoman style app/ working directory and dist/ production directory. All it does is copy the javascript and html from the app/ directory to the dist/ directory.

Let's look at that paths object we're defining. That's not something specific to Gulp - remember, we're just writing JavaScript here. We can define any sort of variable we like and use it however. Just notice the actual path strings I'm setting. They work just like in Grunt, so ** is a wildcard meaning this and any subdirectory, and * is a typical wildcard for the filename. Prepending a path with an exclamation mark tells Gulp to exclude that directory.

In Gulp, you define the instructions for each task in a function passed as an argument to the task. gulp.src takes an array of source paths. gulp.dest copies results to a given directory. And pipe chains tasks together.

Note how I'm concatenating the two paths arrays we defined in paths. Again, you might be used to being stuck in the confines of Grunt's configuration specifics, but here we're just passing a parameter to a function. gulp.src accepts an array of paths, so we can pass that however we want.

Try this gulpfile out if you want, and you'll see the specified .html and .js files in the app/ directory will be copied into the new dist/ directory.

A Real Gulpfile

Let's take a look at one more example, this time something you might find in a real app. The main idea for this gulpfile is similar to our above example; we're still building development files from an app/ folder into a production dist/ folder, we're just doing a little bit of processing in between.

We're using five new plugins this time around: gulp-clean, gulp-jshint, gulp-concat, gulp-uglify, and gulp-imagemin. You can see the full list of plugins in the plugin directory on the Gulp site. Just like in Grunt, when you use a new plugin in Gulp, you'll have to install it with npm. So to install these plugins and write them to your package.json, use something like this:

npm install --save-dev gulp-clean gulp-jshint gulp-concat gulp-uglify gulp-imagemin

Notice the default task definition down towards the bottom of this gulpfile. You can define a task as a sequence of other tasks by passing an array of task names instead of a function. Here, the default task runs clean, scripts, imagemin, and copy, so let's take a close look at each of these.

gulp-clean

This plugin will simply delete files for you, allowing you to get rid of old or transitional files. The first thing this gulpfile does is run gulp-clean on the production dist/ directory, just to make sure that nothing created in a previous build and later removed in app/ could be lingering around.

scripts

The scripts task is going to lint and uglify all of the app's JavaScript, concatenate it all into one file, and output it into our production directory.

gulp-jshint

jshint is a great tool for catching errors and bad habits in your code, and this plugin just runs it on the given files. Note that we also need to use a reporter here if we want to see jshint's output on the command line. jshint.reporter('default') is fine if you're not picky about what it shows you, but go ahead and look at the Github repo if you want to get more specific. And don 't forget that this plugin will be looking in the root of your project for a .jshintrc file to get your configuration preferences.

gulp-uglify

This plugin will minify your code by removing extra whitespace and other stuff that only matter to a human reader, not the browser. Pipe in the files you want it to operate on like normal.

gulp-concat

This will concatenate all given files into one file to reduce the number of HTTP requests required to get your code. Just pass in the filename that you want it output as. Note that you should still add a gulp.dest to the end of this to actually output the file into a given directory like normal.

gulp-imagemin

This plugin reduces the weight of your images. Here it's used with gulp.src and gulp.dest like normal to output the final files to the production folder.

Multiple Instructions in One Task

The copy task is very basic, just like in the previous example. Notice that this time we're creating multiple sets of instructions in one task, copying independent gulp.src files to independent gulp.dest destinations within the task function. This is perfectly OK in Gulp. Once again, it 's just JavaScript.

cwd Option

Note the cwd parameter I'm passing to gulp.src in most tasks. The basic usage of this does what you might expect, changing the working directory of the task so you don't have to repeat redundant subdirectories on all of your paths.

It can also be used to tell Gulp to preserve your directory structure when outputting files, however. If you use the directory wildcard in your cwd like app/** in the libs portion of the copy task, Gulp will compare the directories it finds there with the paths provided in the base path in the first parameter, and use this to reconstruct your directory structure in the output. If this is omitted, all of your output files will be flattened into a single directory level.

watch

The last thing to notice in this gulpfile is the watch task. This watches the given path for any files to change, and executes the array of tasks when they do. This is great for when you're in the middle of development. It's a pain to switch to the terminal and run gulp every time you make a small change, so set up a watch task like this to save you the trouble. It's not included in the default task here, so keep in mind you would run it with gulp watch from the command line.

Tasks in Parallel and Series

A big caveat to watch out for when using Gulp is that all tasks are executed in parallel by default. This is very useful, but can cause problems if you're not careful.

In the gulpfile above, our clean task must complete before any of our tasks that output files into the dist/ folder (scripts, imagemin, and copy). We indicate that by passing ['clean'] in as the second parameter to the definitions of these tasks. This parameter is an array of any tasks that need to complete before the task being defined is run, so you can include as many as you want.

Just make sure that your task that you want to complete first returns the Gulp stream (which is the result of a Gulp operation) when it completes, as done above in the clean task. This tells Gulp that it has completed. You can also take a callback as the parameter to your task and call it when the task completes, in case you aren't generating a stream. The documentation explains this in detail

From Here

You should be able to set up the build process for a sizeable app with this guide alone, but I hope you don't stop there with learning Gulp. Don't forget to refer to the Gulp plugin directory to get ideas on how you can use Gulp to automate much more than is covered in this article. There are already hundreds of plugins available, but Gulp is still lagging far behind Grunt in this area. If you're feeling really adventurous, check out the documentation on creating your own plugin!

Update 2014.04.13 I added the above clarification about tasks in parallel and series due to running into problems with this in my own Gulp projects.

Announcing Mathpx

Mon, 17 Feb 2014 17:29:25 -0800

Ever since I worked for a startup building educational kids apps in China, I've been tossing around a few ideas about writing one of my own. I may be biased as an engineer, but one of the main areas I perceive to be lacking both in the classroom and the app store is our ability to teach math. Mathpx is my new experiment to rethink the way we teach the very fundamentals of arithmetic.

Math not Memorization

When I was an undergraduate in electrical engineering at the University of Michigan, I took quite a few math-heavy courses, and I quickly discovered the obvious fact that you're a lot better off in a class when you understand the fundamentals as opposed to just regurgitating formulas.

Some of my first rigorous math classes really made me realize how much I hadn't understood of the basics in grade school. I had breezed through the Nevada public school system answering questions like "What is pi?" with "3.14" without realizing that was only the tip of the ice berg.

With Mathpx, I wanted to help kids get a solid foundation in the most basic concepts possible. Based on my own grade school experiences and my observations volunteering in classrooms, I realized that I had to take it way back.

1 + 1 = Disconnect

Enough has been said elsewhere about education systems' reliance on memorization and testing, but the way we teach math doesn't just encourage memorization, it prevents anything deeper from day one.

Why do we teach our children subtraction in the first grade but wait years to introduce the concept of negative numbers? 2 - 1 is something a toddler should be able to calculate, but change the order to -1 + 2 and that's taboo until middle school, and 2 - 1 = x has to remain hidden for even longer.

Each of these problems are fundamentally the same thing, but with our grade school they require a leap in understanding to progress. Why not just teach the fundamentals at the start?

We're not Limited to Counting on our Fingers Any More

With the fairly recent rise of tablets, we're just getting started with what teachers can do when they're not limited to counting apples on their hands and writing problems on the board. I scrapped all of this and tried to rethink what the connections to basic arithmetic really are, and this is what I came up with.

Mathpx

In Mathpx, players count out the answers to basic arithmetic problems by dragging balls around the screen. The written form of the problem sits right above and automatically updates, making the connection to pencil-and-paper math intuitive.

Pressing the equals button racks up all number balls, so it's easy to see if the two sides of the equation are equal or not.

Let's make the jump to subtraction. Subtraction works the same way, but it shows numbers being subtracted as blue balls instead of red. Now when the player presses the equals button, positives and negatives visually cancel out, and the remaining values rack up.

With this visualization of math, why hold out on negatives for the next few years? Here is the easy jump to negatives.

Mathpx is designed to evoke a visualization of math that fits nicely with everything from counting to arithmetic with positives and negatives to algebra, because really, it's all one concept.

A Challenge

Don't believe me that your kid isn't ready to learn this level of math right now? Hand them a device with Mathpx on it and see what happens. When you're teaching the fundamentals from the start, kids can learn in an afternoon play session what the school system thinks should take years.

The App

Mathpx is just a hobby experiment of mine, so you can play it for free right now in the browser at mathpx.justinmccandless.com. It's also live on the Chrome Webstore and the Firefox Marketplace, also for free.

Hopefully I'll get around to releasing this on Android and iOS too, and if so I will update this page. In the meantime, enjoy the app, and here's to better ways of teaching math.