Atom Text Editor Tips, Tricks, and Shortcuts

In February of 2014, Github announced Atom as a new text editor built on top of web technologies. I tried it right away but found it lacking compared to what Sublime Text offered.

Over the past few months I’ve given it another go and discovered that in the year that passed, Atom has grown up significantly and, in most ways, even surpasses Sublime Text.

This post is going to look at tips and shortcuts to provide maximum efficiency inside of Atom.

General Tips

Command Palette

The command palette isn’t unique to Atom, but it’s the first tip here because of its sheer power and the way it keeps you from ever having to leave the keyboard.

To open the command palette, use the keyboard shortcut cmd-shift-P (Mac) or ctrl-shift-P (Windows and Linux).

From here you can do a ton of stuff, from opening settings to toggling word wrap. There are too many options to list here, so best thing to do is to open up the command palette next time you find yourself going for your mouse. See if you can get it done with just the keyboard instead.

Open Atom from Command Line

From anywhere in the command line, you can open a file or a directory by using the atom command. For example, to open the current directory:

1
$ atom .

Swapping Lines

Swapping lines is usually one of the first “wow&rqduo; moments someone who is unfamiliar with text editors have. It’s one of the tools that makes Atom powerful not just for code editing, but for general writing, too. You’ll find yourself wanting it in other programs, like your email client.

To swap lines up, the keyboard shortcuts are:
ctrl-cmd-up (Mac)
ctrl-up (Windows and Linux)

Here it is in action:

Autoindenting While Swapping Lines

This isn’t something you have to actively do, but it’s a neat feature that’s built into the text editor.

When you are swapping lines and you enter a block of indented code, Atom will match the indentation of that block.

Snippets

With snippets, you can create text expansion once you type a few keystrokes and hit tab. Even better, you can scope snippets to certain file types.

To create your own snippets, go to Atom > Open Your Snippets. This will open ~/.atom/snippets.cson, a CoffeeScript Object Notation file. CSON is just like JSON, but CoffeeScript. So watch your whitespace.

If I wanted to create a snippet to add my email address, I would add the following:

1
2
3
4
'.source.gfm':
'Email':
'prefix': 'email'
'body': 'dustin@dcoates.com'

The first key is the scope. Here, gfm stands for GitHub Markdown. My email snippet will then only be available inside of that syntax.

Be careful about scopes: if you duplicate one inside the snippet declaration file, only the last one will take hold.

The next snippet is the description. This can be multi-worded if you want.

Next is the prefix. This is what you want to type that will trigger the completion.

Finally is the body. In this simple example, what is expanded is just my email address.

You can put cursor points in, however. These are places where, after the text is expanded, you have the opportunity to add your own text by tabbing through. To do this, you’ll use the $N notation.

1
'body': '<input type="text" placeholder="$1" autocomplete="off">'

When you expand the preceding snippet, your cursor will be within the placeholder attribute so you can add your own.

You can also add defaults, using ${N:default}.

1
'body': '<input type="text" placeholder="${1: Search}" autocomplete="off">'

One last thing: if you want to add a longer snippet across multiple lines, use """.

1
2
3
4
5
'body': """
<div>
$1
</div>
"""

You can, at any time, show the snippets that are available to you with alt-shift-s.

TODO and FIXME Highlighting

Another useful feature that Atom has built in is highlighting of TODO and FIXME.

This provides an easy way to scan in-progress code and see what’s left to be done.

In addition to those two, Atom also will highlight XXX, CHANGED, IDEA, HACK, NOTE, and REVIEW.

Paired with a package (see below), this ensures you aren’t forgetting to do something you had put off in the last-minute rush for release.

Markdown Preview

A useful tool that is built into Atom is the ability to preview your markdown. This means that Atom is not only a powerful tool for writing code, but it’s powerful for writing text as well. And unlike other markdown editing tools, it’s free and not reliant on a single developer.

To bring up the markdown preview pane, use the keyboard shortcut ctrl-shift-M or use the command palette.

Version Control

Coming from Github, Atom ships with git integration beyond what you’ll find as default from other text editors.

You can use the file switcher (cmd-p (Mac) and ctrl-p (Windows and Linux)) to quickly see which files have been changed in the current state.

New files are denoted with blue and changed files with gold (at least in my color scheme).

You can bring these files up even quicker by bypassing the normal file switcher and bringing up just new and changed files with the shortcut cmd-shift-B (Mac) and ctrl-shift-B (Windows and Linux).

Interfile Tracking

That’s nice to have, but I find tracking changes within a file indispensable.

Here you can see the three types of indicators. Again, the colors are according to my current scheme, but they’re pretty standard:

  • Green: new line
  • Red: deleted line
  • Gold: changed line

In the lower-right hand corner of your window, you can also see the number of changes in your file (additions and deletions), as well as the current working branch.

Deep Github Integration

You didn’t think there would be a Github text editor without even deeper Github integration, did you? If your current project is on Github, you can:

  • Open the current file with current line number highlighted with alt-g o
  • Blame the file with alt-g b
  • Open history with alt-g h
  • Open issues for the repo with alt-g i
  • Open the repo itself with alt-g g
  • Copy the URL for the file in with alt-g c
  • Create a pull request for the current branch with alt-g r

Truth be told, I find these keystrokes to be a bit awkward to type. Thankfully, like most of the actions in Atom, you can get at all of these through the command palette.

We’ve already seen how to navigate between files (cmd-p and ctrl-p), but the navigation goes well beyond that.

You can go to a specific line and column within the current file with the shortcut ctrl-g and then specifying the line and column, separated with a colon.

This is another shortcut I find cumbersome, typically preferring to go through the command palette.

Going to a line and column in another file is, thankfully, much easier. Bring up the file switcher (cmd-p/ctrl-p), type enough to bring up the desired file, then without spaces type the line number, colon, and the column number.

Navigate symbols within a file with the cmd-r (Mac) or ctrl-r (Windows and Linux) shortcut.

Navigate symbols throughout your project with cmd-shift-R or ctrl-shift-R.

To navigate symbols globally, you must have tags generated for your project through ctags.

There’s a lot more to it, but that’s covered in loads of other tutorials on the web. I hope you learned something new that brings your Atom usage to new levels.

252 Web Developer Career Resources

This was originally published on the Pico website. I am republishing it here.

Tracking your progress as you make your way in your career as a web developer is important, but that can be tough if you don’t know the steps you need to take to go from, say, a beginner-level dev to a junior-level dev. That’s why we created this list and provide you resources for every step along the way. This list is in-depth, but if you think I missed something, send it along.

Junior

This level is by far the largest, as you’re just getting started. Don’t let the breadth of the list scare you, however. As you progress to being a junior developer, you will want to touch on a lot of basics which will help you as you get to other skills on the list. Think of it like one of those cartoons where a snowball rolls down a hill, quickly growing as it does. That will be you! So get in there and start building your skills.

Computing Basics

To know how to write programs with a computer, you need at least a basic understanding of how computers work. No need to get into the silicon itself, but these resources provide a high-level (and, in some cases, deeper) overview of what’s going on with your machine.

HTML

The markup of the web. If you are going to work in web development, even as a back-end developer, you have to know HTML in and out. The good news is that it isn’t that difficult and there are a multitude of resources out there to help you get started, like the ones listed here.

CSS Fundamentals

What makes the web beautiful. CSS is easy to learn, but can take you forever to master. Focus on the basics for now and if you decide that being a front-end expert is the path you want to take, CSS is rapidly growing and offers you a lifetime of learning opportunities.

JavaScript Fundamentals

While long looked down on by programmers who saw the language as little more than a toy, JavaScript has shown itself to be anything but. On the front-end, you can use JavaScript to make fast, engaging web apps. And on the back-end, use JavaScript with Node or Io.js. JavaScript is extremely in demand these days and it’s easy to see why.

Command Line Fundamentals

Your mouse can only take you so far. Whether you use Windows, Mac, or Linux, learning the command line is like a power-up for your productivity. Do things in seconds that would take you minutes or hours otherwise.

Learning Your Text Editor or IDE

Your choice of text editor or IDE is a personal one so we won’t tell you which to go with. But we will say that getting deeply into the features and tools of whatever you choose is immensely valuable. When you look at senior devs moving around so fast it’s because they’ve invested time in learning their tools. Start with these resources below.

Basic Data Structures

This is what all the programming languages are built on. You’ll learn most of this as you learn the different programming languages, but if you want a shortcut, look at these resources.

Data Modeling

Spending just a little bit of time up front to model your data will save you immensely in the long run. Check out this course to get the basics.

API Basics

APIs are a must-learn skill for all programmers. If you work on the front-end you will be consuming APIs and if you work on the back-end you will likely be consuming and creating APIs. As a junior programmer you don’t need to go too far in-depth into APIs, but learn the basics with these resources.

Version Control

It’s difficult to imagine how teams of programmers worked together before version control. Version control allows us to not only collaborate with others, but to be more efficient when working by ourselves. Git is the behemoth in version control, so most of the resources here deal with it. But we also have resources discussing version control from a high level that will get you started.

Database Fundamentals

Databases are the first differentiation between a static webpage and a website that users can really interact with. Learn the basics of databases and understand the difference between SQL and NoSQL databases with these guides.

TDD

TDD (Test Driven Development) is a contentious topic in modern programming, which is why you should be aware of the different arguments in favor and against TDD. Here we provide both points of view.

Debugging

Before you become a programmer, you don’t realize how much of your time is spent fixing things that have gone wrong. You assume that as you become more of an expert you do less of it, but debugging is an invaluable skill for developers of all skill levels.

Professional Skills

As much as we would love for it to be the case, you can’t be a truly successful web developer without strong professional skills. These books are considered classics for a reason: they will help you build a bedrock for being a web development professional.

Communication

Like any other job, communication is key as a web developer. The way you communicate with other developers, your bosses, designers, business people, and customers will go a long way toward determining the level of success you’ll have in your career.

Code Review

Providing and receiving feedback on your code is an important part of working on a development team. Use these resources to learn best practices for code review.

Regular Expressions

Although they can seem impenetrable when you’re starting out (and sometimes even beyond), regular expressions can be immensely valuable when you need them. Think of them as a little something extra to your toolbox—you’ll never be hired as a Senior Regular Expressions Developer afterall.

Back-end

Algorithm Fundamentals

Algorithms are the difference between code that finishes in half a second and code that times out. Learn these to really take your skills to the next level.

SQL

With fancy ORMs like ActiveRecord and SQLAlchemy, you might think SQL is a skill you no longer need to master. Until, of course, you have a complicated query that has to run efficiently and you need SQL to do so.

DevOps

You could spend your entire career in ops if you wanted to. But even if you decide that’s not the path for you, having a grounding what’s going on is important.

Computer Science Fundamentals

If you don’t come from a computer science background, you might feel that it will always be out of your reach. The good news is that there are so many resources online that you don’t need to feel that way. You may not get the diploma, but you’ll have the knowledge.

Front-end

From interacting with the DOM to extending JavaScript’s capabilities, these are the canonical JavaScript libraries.

Intermediate JavaScript

As a prototypical language and owing to its history of being the programming language for the web, JavaScript is easy to get started with but has some interesting surprises for you when you get to the intermediate level. These resources will help you navigate that chasm.

JavaScript MVC Frameworks

Backbone, Ember, Angular, none of the above? Dabble with the major JavaScript MVC frameworks and decide which you like best and work best for your projects.

Responsive Web Design

Smartphones are taking over the world and there’s no excuse for designing for the desktop only anymore. Learn responsive web design and have happier website visitors.

Front-end Performance

Webpage load time matters a lot. Don’t create a bloated site that drives your users away—learn how to make your front-end fast.

Front-end Tooling

The toolset for front-end development was long a barren area. While it hasn’t caught up to the vast tools available to those on the back-end, it’s catching up quickly. Choose between SASS, LESS, and Stylus or decide between Grunt and Gulp. Half the challenge is deciding which tool to use.

CSS Strategies

As our sites get larger and larger, the amount of CSS we use grows as well. Learn about strategies to keep your CSS tidy and maintainable.

Browser Developer Tools

You’ll be surprised at just how powerful the developer tools inside your browser can be. While the resources here are Chrome-heavy, Firefox is improving their developer tools daily.

Mid-level

As you’re headed to mid-level status as a dev, you’ve built a good foundation and now it’s time to go deeper. But don’t let that keep you from trying out new technologies. Being well-rounded is an important part of any dev job and even if you don’t use your new skills in your day-to-day job (maybe you are a Ruby dev by day but learn Scala by night), you’ll be able to have a new insight into the code you write.

All

Refactoring

The best laid plans… Keep these strategies in mind when your code has become unruly or the needs of the project have changed.

Back-end

Design Patterns

Don’t reinvent the wheel. Being a student of programming design patterns will help you better tackle challenging problems when coding.

Service Oriented Architecture

Service Oriented Architecture and microservices have exploded in popularity and for good reason, but doing it wrong can be worse than just keeping a monolithic architecture. Learn the pros and cons of SOA, plus how to go about it right way and when.

API Design

You know how to consume APIs, but do you know how to design APIs that are extendable and other developers will find a joy to use?

Performance Testing

Don’t let the burden of fast-loading websites fall on your front-end compatriots alone. You have a role to play, too. Look into these resources on how to make sure your website is not only fast but can withstand getting on the front page of Hacker News.

Front-end

UI Components

Many smart people think web components are the next big thing for front-end development. Whether through native components, Polymer, or React, you’ll be well-served to learn more about this rising technology.

JavaScript Modularity

One of the big complaints about developing with JavaScript is that it has been difficult to maintain modularity. Thankfully, tools are here that help us achieve that. Check out some of them below.

JavaScript Testing

Testing hasn’t always been at the forefront of JavaScript development. That has changed in recent years and as a mid-level developer, you should know how to test your JavaScript, no matter what testing framework you use.

Automated UI Testing

The last thing you want is to find an email from a customer on a Saturday evening that a page that you forgot to check is now broken as a result of some CSS changes you made the previous week. Automated UI testing is still in its infancy, but helps insure that you avoid these kinds of side effects.

Senior

This is the shortest section as, by now, you are really deep into your craft and you are smart to focus on becoming an expert in just a few specializations. Since we can’t list the best resources for every programming language out there, we focus instead on the new skills you’ll need as a senior level developer. These aren’t technical skills—they’re people skills. As a senior developer you will want to lead and influence people and the people side of that is incredibly important.

All

Leadership

You’re now in a position where people look up to and take cues from you. While this might be scary at first—you can’t re-run interactions with people with different inputs like you can with code—you can hack your way to being a better leader. The books below are a great place to start.

Mentorship

Here’s your opportunity to be a multiplier. When you mentor those more junior than you, you raise the level for everyone. Even better, you’ve likely heard that one of the best ways to learn is by teaching and it’s true. Look to the resources below for how to be the best mentor you can be.

Quick JavaScript File Generator for Fish

I’ve found that I follow the same steps to create a new JavaScript file every time: touch the new file, then add "use strict"; to the top of it.

Luckily, fish shell makes it easy to create functions to streamline repetitive tasks.

1
$ function strict -a path
  echo '"use strict";' > $path
end
$ funcsave strict

The -a flag tells the function to accept an argument named path. Simple script to create a JavaScript file with "use strict"; at the top.


Find this interesting? Follow me on Twitter and let’s discuss alternative shells and side projects.

Project #4: An Amazon Echo Skill

At the end of 2014, Amazon announced a new product–a voice-powered and wireless-enabled speaker called the Echo–that seemed a little strange for them. Amazon had ventured into hardware before with the Kindle, the Fire, and the ill-fated Fire Phone but the Echo didn’t seem to fit into Amazon’s goal of cheap hardware to enable people to buy more stuff. Afterall, initially the only thing you could buy through the Echo was music.

Amazon kicked off the Echo with invitation-only purchasing for Prime members, plus a $100 discount (bringing the price to $99) and a lengthy waitlist even after you purchased. Indeed, I knew people who purchased an Echo in January only to not receive it until May.

At first the Echo was a nice speaker, especially for the price, but nothing to really write home about. And it didn’t take advantage of the cloud-enabled features to the fullest extent. Then in late spring Amazon announced quietly that they would open up the Echo to developers to build upon.

The Alexa Skills Kit is a set of SDKs that enable developers to build voice-powered “skills” for the Echo. This move:

  • Gives developers SDKs in Java or JavaScript
  • Allows developers to host their own endpoints that the Echo would speak to or leverage the AWS Lambda compute service
  • Provides a structure for easily tapping into voice commands

The Project

For this project, I built a skill that probably doesn’t have wide appeal but is nonetheless one that I need. This skill allows the user to specify a New York City bus stop by its unique stop number (e.g. 308638) and hear spoken back how far the next bus is away. I always try to leave my apartment when the bus is 0.6 miles away from the stop so this will help me know when I should start packing up.

I’m disappointed to say, but developing for the Echo using Lambda was a total pain. If you need more than one file (and of course you do, because you’re not building a trivial skill, and any way, using the JavaScript SDK puts you at two files at a minimum) you have to compress and upload your files every time you make a change. The user interface leaves a lot to be desired, a common problem with AWS. Logging isn’t very robust or usable though it is built in, so that’s a plus.

On the other hand, being able to direct what Echo’s voice-avatar Alexa speaks back to you is fun. And thinking about how the user is going to interact with your skill is an interesting user experience challenge that most developers rarely get a chance to consider.

The Guide and Tutorial

I changed up my approach a little bit this week and decided to go deeper into the details, writing a guide to the Amazon Skills Kit JavaScript SDK and a tutorial for developing an Amazon Echo skill. Check those out or check out the code on Github.


Find this interesting? Follow me on Twitter and let’s discuss developing for the Amazon Echo and other side projects.

Amazon Alexa JavaScript SDK: The Ultimate Guide

Hey! This guide is still useful, but to get all the most up-to-date information on building Alexa Skills, follow my newsletter/blog about voice-first platforms.

And if you’re interested in a shortcut for building Alexa Skills, particularly with full-text search, give this Alexa Skills full-text search from Algolia adapter a try.

In this project, I’m going to walk you through the Amazon Alexa Skills Kit JavaScript SDK. Completely unfamiliar with the Alexa Skills Kit, Amazon Echo, and coding for them? Check out my tutorial for developing an Amazon Echo skill.

Download the SDK and follow along.

We’re going to go through the code from top to bottom and explain each section.

1
2
3
function AlexaSkill(appId) {
this._appId = appId;
}

This is our constructor. Most of the rest of the SDK is adding methods onto this object’s prototype. We’re going to extend this constructor when we create our skill. It accepts an appId argument that you get when you create an Alexa skill inside the Amazon Developer Console.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
AlexaSkill.prototype.requestHandlers = {
LaunchRequest: function (event, context, response) {
this.eventHandlers.onLaunch.call(this, event.request, event.session, response);
},

IntentRequest: function (event, context, response) {
this.eventHandlers.onIntent.call(this, event.request, event.session, response);
},

SessionEndedRequest: function (event, context) {
this.eventHandlers.onSessionEnded(event.request, event.session);
context.succeed();
}
};

The first two methods are really just syntactic sugar for two different event handlers. The third is invoked when the session ends and signals that the request has succeeded. You won’t really find yourself using these methods directly–these are primarily used inside the execute method we’ll look at later.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
AlexaSkill.prototype.eventHandlers = {
onSessionStarted: function (sessionStartedRequest, session) {
},

onLaunch: function (launchRequest, session, response) {
throw "onLaunch should be overriden by subclass";
},

onIntent: function (intentRequest, session, response) {
var intent = intentRequest.intent,
intentName = intentRequest.intent.name,
intentHandler = this.intentHandlers[intentName];
if (intentHandler) {
console.log('dispatch intent = ' + intentName);
intentHandler.call(this, intent, session, response);
} else {
throw 'Unsupported intent = ' + intentName;
}
},

onSessionEnded: function (sessionEndedRequest, session) {
}
};

We’ve got four methods. Let’s examine them one-by-one.

onSessionStarted(sessionStartedRequest, session)

This is called when the session starts. It’s optional and used for any set up your skill needs.

onLaunch(launchRequest, session, response)

This is the only method that you’re required to override. This method is invoked when a user launches a skill without specifying an intent. Generally what you’ll want to do here is prompt the user to specify their intent.

onIntent(intentRequest, session, response)

When a user launches your skill and specifies what they want. You will most likely not override this method. The first argument is an object that has information about the request. We grab the intent off this request and look to the intentHandlers object to see if we have a handler for the given intent. If we do, we call the handler, otherwise we throw an error. We’ll look at the intentHandlers object in a bit.

onSessionEnded(sessionEndedRequest, session)

This is called when the session ends. Just like the onSessionStarted method, overriding this method is optional.

1
AlexaSkill.prototype.intentHandlers = {};

This is an object that will hold your intent handlers. These will be the same intents that you define in your intent schema inside the Amazon Developer’s Console.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
AlexaSkill.prototype.execute = function (event, context) {
try {
console.log("session applicationId: " + event.session.application.applicationId);

if (this._appId && event.session.application.applicationId !== this._appId) {
console.log("The applicationIds don't match : " + event.session.application.applicationId + " and "
+ this._appId);
throw "Invalid applicationId";
}

if (!event.session.attributes) {
event.session.attributes = {};
}

if (event.session.new) {
this.eventHandlers.onSessionStarted(event.request, event.session);
}

var requestHandler = this.requestHandlers[event.request.type];
requestHandler.call(this, event, context, new Response(context, event.session));
} catch (e) {
console.log("Unexpected exception " + e);
context.fail(e);
}
};

The real meaty part of the code. This method is only invoked in the exported handler inside of your code in AWS Lambda, like so:

1
2
3
4
exports.handler = function(event, context) {
var skill = new BusSchedule();
skill.execute(event, context);
};

The first conditional checks that the applicationId on the session matches the appId we set in constructor.

The second conditional ensures that the session attributes is an object. And the third conditional invokes the onSessionStarted event handler if the session is marked as new.

Finally, we grab the request handler for the request’s type and calls the handler.

Of course, if at any point the code throws an error, the error will be logged and the fail method is invoked on the context object.

The Response Object

1
2
3
4
var Response = function (context, session) {
this._context = context;
this._session = session;
};

Here’s another constructor, this time for the response, which accepts a context object and a session object. You’ll never instantiate with the response object directly–it’s passed through to the request handlers inside the execute method.

This object returns four methods: tell, tellWithCard, ask, and askWithCard.

The Tell Methods

tell(speechOutput)

tellWithCard(speechOutput, cardTitle, cardContent)

We have two methods here which will respond to the user and end the session. First is tell, which accepts a string that Alexa will speak to the user, and tellWithCard, which accepts a string that Alexa speaks to the user, a string that serves as the card title, and a string that serves as the body of the card. The card is displayed within the Amazon Echo app.

The Ask Methods

ask(speechOutput, repromptSpeech)

askWithCard(speechOutput, repromptSpeech, cardTitle, cardContent)

These methods are just like the tell methods, except for two key differences. First, the session is kept open, waiting for a further response from the user. Second, the second argument is a string that Alexa will speak to the user if they haven’t responded to specify what they want.

The Response Object

These methods all assemble an object. Here’s a fully assembled object, with notes for each property:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
{
version: '1.0',
response: {
outputSpeech: { // Always part of the response
type: 'PlainText', // Always this value
text: 'Example speech' // Text that will be spoken back to user
},
shouldEndSession: true, // Always part of the response--boolean for if session has ended (no more user input)
reprompt: { // Optional, if you want to customize what Alexa says if user doesn't respond correctly/at all
outputSpeech: {
type: 'PlainText',
text: 'Example reprompt'
}
},
card: { // Optional, if you want to show a card in the Echo app
type: 'Simple',
title: 'Example title',
content: 'Example card content'
}
},
sessionAttributes: {} // If you need some attributes to be kept in the session
}

Note that you do not have to assemble this object yourself–the response methods will do it for you.

This is a look at the Amazon Alexa Skills Kit JavaScript SDK. It packs a lot of functionality into less than 180 lines of code. If you want to check out the code more in-depth check it out here.


Find this interesting? Follow me on Twitter and let’s discuss developing for the Amazon Echo and other side projects.

Alexa Skills Tutorial: The Definitive Guide to Coding for the Amazon Echo

Hey! This tutorial is still useful, but to get all the most up-to-date information on building Alexa Skills, follow my blog about coding for the Amazon Echo, Google Home, and other conversational devices.

And if you’re interested in a shortcut for building Alexa Skills, particularly with full-text search, give this Alexa full-text search from Algolia adapter a try.

In this tutorial I’m going to walk you through how to create a “skill” for the Amazon Echo. The Echo is a voice-powered and wireless-enabled speaker from Amazon that was released in late 2014. A skill can be thought of as an app that increases the capabilities a user can take advantage of with the Echo.

The skill that we’re going to be building is a way for users to ask Alexa, the voice avatar behind the Echo, when the next bus will arrive at a New York City bus stop. To keep this simple and to focus on the Echo side of the equation, we’re going to require users to know the stop ID that they want. This means that they will have to request “stop three zero eight six three eight” and not “Myrtle at Flushing.”

Get a Key from the MTA

The first thing we need to do is request a developer key from the MTA. This is required for our API calls and can rarely (but sometimes) take up to thirty minutes to get a key, so let’s start off the process.

First go here and following the link to register for a key. The website says your application needs to be reviewed and that it will take up to thirty minutes to get your key. That hasn’t been my situation, but be aware. If it does take a little bit of time for you, we can get started while you wait.

Setup a Skill

From here we want to create a Lambda function. Lambda allows our first million invokations a month to be free and Amazon requires any code we would use on our own server to be behind https, so Lambda is the ideal approach for building an Alexa skill.

Log in to AWS and select Lambda. Make sure that you’re in the N. Virginia region, as Alexa Skills are currently only supported there. On the next page, click on the big button that says “Create a Lambda function.”

Amazon provides you with a number of blueprints that you can use to build your Lambda function and, since we’re going to be building an Alexa skill, select alexa-skills-kit-color-expert.

The event source type should have pre-selected Alexa Skills Kit. If not, select that and move on the next section. Give your function a name and a description (this is just for you, so name it something memorable). Leave the code that’s in there. We’ll come back and replace it later.

The handler we want to keep as index.handler. What this means is that Lambda is going to look for an exported function named handler inside of a file named index.js.

For the execution role, select Basic execution role. In the window that pops up, just go ahead and click “Allow.” You can then leave the memory and timeout settings the same before moving on to the next screen, reviewing everything and clicking on “Create function.”

On the next page, you should see on the right-hand side a string that starts off with “arn:” and ends with your function name. Note this, because we’ll need it in our next step.

The next thing we want to do is set up our Alexa Skill. Start by going to the Alexa Console and logging in. On the next page, select Alexa Skills Kit.

On the next page, select “Add a New Skill.” This will take you to a page where you can name your skill (which will be shown in the Echo app), create your invocation name (what users will say to start your skill), and the endpoint that we got when we created our Lambda function.

We’re going to name our skill Bus Schedule and use bus schedule for our invocation name. Add your Lambda function endpoint and submit the form.

The next screen asks us for our intent schema and sample utterances. The intent schema is JSON that defines our intents (the actions that user can take within our skill) and slots (our arguments within a given intent). The sample utterances are example invokations of our skill. It provides the Echo backend a guide to how the user will interact with our skill.

Here is our intent schema:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
{
"intents": [
{
"intent": "GetNextBusIntent",
"slots": [
{
"name": "bus",
"type": "NUMBER"
}

]
},

{
"intent": "HelpIntent",
"slots": []
}

]
}

This defines two intents. The first is the GetNextBusIntent with a type of NUMBER. The other types available are DATE, TIME, DURATION, and LITERAL. The first three are straight-forward and LITERAL is just everything else. The second intent has no slots and will be available for users to ask for help on our skill.

Here are our sample utterances:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
GetNextBusIntent get next bus for {three zero four four four two|bus}
GetNextBusIntent get next scheduled bus for {three zero four four fiv six|bus}
GetNextBusIntent get next arrival for {five zero eight nine zero five|bus}
GetNextBusIntent get bus for {two zero eight nine seven nine|bus}
GetNextBusIntent next bus for {three zero four four four two|bus}
GetNextBusIntent next scheduled bus for {three zero four four fiv six|bus}
GetNextBusIntent next arrival for {five zero eight nine zero five|bus}
GetNextBusIntent bus for {two zero eight nine seven nine|bus}
GetNextBusIntent get next bus at {three zero four four four two|bus}
GetNextBusIntent get next scheduled bus at {three zero four four fiv six|bus}
GetNextBusIntent get next arrival at {five zero eight nine zero five|bus}
GetNextBusIntent get bus at {two zero eight nine seven nine|bus}
GetNextBusIntent next bus at {three zero four four four two|bus}
GetNextBusIntent next scheduled bus at {three zero four four fiv six|bus}
GetNextBusIntent next arrival at {five zero eight nine zero five|bus}
GetNextBusIntent bus at {two zero eight nine seven nine|bus}
GetNextBusIntent get next bus {three zero four four four two|bus}
GetNextBusIntent get next scheduled bus {three zero four four fiv six|bus}
GetNextBusIntent get next arrival {five zero eight nine zero five|bus}
GetNextBusIntent get bus {two zero eight nine seven nine|bus}
GetNextBusIntent next bus {three zero four four four two|bus}
GetNextBusIntent next scheduled bus {three zero four four fiv six|bus}
GetNextBusIntent next arrival {five zero eight nine zero five|bus}
GetNextBusIntent bus {two zero eight nine seven nine|bus}
GetNextBusIntent find next bus at {three zero four four four two|bus}
GetNextBusIntent find next scheduled bus at {three zero four four fiv six|bus}
GetNextBusIntent find next arrival at {five zero eight nine zero five|bus}
GetNextBusIntent find bus at {two zero eight nine seven nine|bus}
GetNextBusIntent get next bus for {five|bus}
GetNextBusIntent get next scheduled bus for {six eight|bus}
GetNextBusIntent get next arrival for {seven nine five|bus}
GetNextBusIntent get bus for {nine zero nine|bus}
GetNextBusIntent next bus for {five|bus}
GetNextBusIntent next scheduled bus for {six eight|bus}
GetNextBusIntent next arrival for {seven nine five|bus}
GetNextBusIntent bus for {nine zero nine|bus}
GetNextBusIntent get next bus at {five|bus}
GetNextBusIntent get next scheduled bus at {six eight|bus}
GetNextBusIntent get next arrival at {seven nine five|bus}
GetNextBusIntent get bus at {nine zero nine|bus}
GetNextBusIntent next bus at {five|bus}
GetNextBusIntent next scheduled bus at {six eight|bus}
GetNextBusIntent next arrival at {seven nine five|bus}
GetNextBusIntent bus at {nine zero nine|bus}
GetNextBusIntent get next bus {five|bus}
GetNextBusIntent get next scheduled bus {six eight|bus}
GetNextBusIntent get next arrival {seven nine five|bus}
GetNextBusIntent get bus {nine zero nine|bus}
GetNextBusIntent next bus {five|bus}
GetNextBusIntent next scheduled bus {six eight|bus}
GetNextBusIntent next arrival {seven nine five|bus}
GetNextBusIntent bus {nine zero nine|bus}
GetNextBusIntent find next bus at {five|bus}
GetNextBusIntent find next scheduled bus at {six eight|bus}
GetNextBusIntent find next arrival at {seven nine five|bus}
GetNextBusIntent find bus at {nine zero nine|bus}

HelpIntent help
HelpIntent help me
HelpIntent what can I ask you
HelpIntent get help
HelpIntent to help
HelpIntent to help me
HelpIntent what commands can I ask
HelpIntent what commands can I say
HelpIntent what can I do
HelpIntent what can I use this for
HelpIntent what questions can I ask
HelpIntent what can you do
HelpIntent what do you do
HelpIntent how do I use you
HelpIntent how can I use you
HelpIntent what can you tell me
HelpIntent how do i find a bus

Let’s break these down. Here’s one as an example:

1
GetNextBusIntent get next bus {three zero four four four two|bus}

This roughly is equivalent to IntentName + spoken statement + slot. The slot can be anywhere inside the spoken statement but it must be in the curly braces with an example of what a user might request first, followed by a pipe, and then the slot name. Here, for example, is a made-up intent that has multiple slots spread out amongst the statement:

1
SetPropertyIntent give {Jane|owner} the property {Park Place|property}

Go ahead and save that–we’ll be coming back later, but for now we need to code up the skill.

Developing the Skill

Let’s first create a couple directories: src and speechAssets. Inside the speechAssets directory, create IntentSchema.json and SampleUtterances.txt. This step is optional, but I like storing a record of the intent schema and the sample utterances alongside the rest of the code. Paste the JSON we created earlier inside IntentSchema.json and the sample utterances inside SampleUtterances.txt.

Inside the src directory, create two files: AlexaSkill.js and index.js. The first file will be for the Alexa Skills Kit SDK, which you can grab here. The index.js file is going to be the main entry point for our skill.

At the top of our file, add the following variable declarations:

1
2
3
4
var http       = require('http')
, AlexaSkill = require('./AlexaSkill')
, APP_ID = 'your-app-id'
, MTA_KEY = 'your-mta-key';

In this section we are requiring the Node package http so we can interact with the MTA API, the Alexa Skills Kit SDK, and specifying our Alexa skill’s app ID plus our MTA key. You can grab the app ID from the Skill Information section where you created your Alexa skill. The MTA_KEY is what you received when signing up with the MTA earlier.

Let’s next create a function that will assemble the endpoint we’ll communicate with to get the bus information from the MTA. It should be pretty self-explanatory:

1
2
3
var url = function(stopId){
return 'http://bustime.mta.info/api/siri/stop-monitoring.json?key=' + MTA_KEY + '&OperatorRef=MTA&MaximumStopVisits=1&MonitoringRef=' + stopId;
};

Our next function will grab JSON from the MTA:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
var getJsonFromMta = function(stopId, callback){
http.get(url(stopId), function(res){
var body = '';

res.on('data', function(data){
body += data;
});

res.on('end', function(){
var result = JSON.parse(body);
callback(result);
});

}).on('error', function(e){
console.log('Error: ' + e);
});
};

This function accepts two arguments: the stopId the user has requested and a callback to handle the data once we’re finished grabbing it.

If you’re not familiar with Node’s http library, we’re listening to the incoming data stream. As soon as it’s finished we parse the string into JSON and pass it to our callback. If there’s an error, we log the error–we’ll look at how to view the logs later.

The next function we create is what will be invoked when the user invokes the GetNextBusIntent, i.e. when they request a bus stop. This function will invoke the getJsonFromMta function we just created and grab just the string we need from the response and send it back to the user.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
var handleNextBusRequest = function(intent, session, response){
getJsonFromMta(intent.slots.bus.value, function(data){
if(data.Siri.ServiceDelivery.StopMonitoringDelivery[0].MonitoredStopVisit){
var text = data
.Siri
.ServiceDelivery
.StopMonitoringDelivery[0]
.MonitoredStopVisit[0]
.MonitoredVehicleJourney
.MonitoredCall
.Extensions
.Distances
.PresentableDistance;
var cardText = 'The next bus is: ' + text;
} else {
var text = 'That bus stop does not exist.'
var cardText = text;
}

var heading = 'Next bus for stop: ' + intent.slots.bus.value;
response.tellWithCard(text, heading, cardText);
});
};

A few things warrant further highlighting here.

The first is intent.slots.bus.value. We are reaching a few levels deep into the intent object and grabbing the bus slot. We know to grab the bus slot because that’s what we named it in our IntentSchema we created earlier.

The next thing we want to do is grab the data that was returned to the callback and traverse the structure. Unfortunately the MTA structure isn’t the most ideal, we need to go nine levels deep before we get what we’re looking for: the PresentableDistance which equates to something like 1.8 miles away, 2 stops away, or Arriving.

Finally, we call the tellWithCard method on the response object. If you’re interested in finding out more about the response object, check out my guide to the Alexa Skills Kit JavaScript SDK. But in this situation, just know that tellWithCard will tell the user the answer (via Alexa) and display a card in the Echo App. The first argument is what Alexa will say, the second argument is the header for the card, and the remaining argument is the body of the card.

Creating Our BusSchedule Constructor

Now what we’re going to do is extend the AlexaSkill SDK and create our own BusSchedule constructor. Again, if you want to dive deep you can check out the SDK guide, but you’ll be able to follow along without reading it first.

Add this code at the bottom of index.js:

1
2
3
4
5
6
var BusSchedule = function(){
AlexaSkill.call(this, APP_ID);
};

BusSchedule.prototype = Object.create(AlexaSkill.prototype);
BusSchedule.prototype.constructor = BusSchedule;

What we’re doing here is creating our BusSchedule constructor and setting its prototype to the prototype from AlexaSkill. This means we get everything that AlexaSkill has and can extend or override as we see fit.

Event Handlers

In fact, we’ll override a method right away. There are four event handlers we can override: onSessionStarted, onLaunch, onIntent, and onSessionEnded. We are only required to override onLaunch. You’re unlikely to override onIntent and onSessionStarted and onSessionEnded are available for setup and tear down.

onLaunch is the event handler that is called when a user has launched the skill but hasn’t specified which intent they want. For example, Alexa ask bus schedule versus Alexa ask bus schedule for the next bus at stop 304442.

1
2
3
4
5
6
7
8
BusSchedule.prototype.eventHandlers.onLaunch = function(launchRequest, session, response){
var output = 'Welcome to Bus Schedule. ' +
'Say the number of a bus stop to get how far the next bus is away.';

var reprompt = 'Which bus stop do you want to find more about?';

response.ask(output, reprompt);
};

All event handlers accept arguments of a request, a session, and a response. Ours is pretty simple–we’re just telling the user what they can do with our skill–but you can make it more complext if you’d like. We’re using the ask method on the response object, which will keep the connection open and listen for a user reply. The reprompt is optional, but allows us to poke a user if they don’t respond soon enough to Alexa’s questioning.

Intent Handlers

The SDK defines an intent handlers object, but leaves it up to you to define the handlers themselves. These correspond to the intents that we specified in our intent schema that we created earlier. In our case, GetNextBusIntent and HelpIntent. Add this to your index.js:

1
2
3
4
5
6
7
8
9
10
11
BusSchedule.prototype.intentHandlers = {
GetNextBusIntent: function(intent, session, response){
handleNextBusRequest(intent, session, response);
},

HelpIntent: function(intent, session, response){
var speechOutput = 'Get the distance from arrival for any NYC bus stop ID. ' +
'Which bus stop would you like?';
response.ask(speechOutput);
}
};

Our first intent handler just invokes our handleNextBusRequest function we created earlier, while our second intent handler tells the user more about the skill we’ve created then keeps the connection open to allow the user to specify their next move.

Finally, we tie all the code together and export our handler:

1
2
3
4
exports.handler = function(event, context) {
var skill = new BusSchedule();
skill.execute(event, context);
};

Uploading Our Code to AWS Lambda

Now we want to put our code up on AWS Lambda. We first need to compress our index.js and AlexaSkill.js files, but make sure you are compressing just the files and not the folder that contains the files. This is really important, otherwise Lambda won’t be able to find your code.

Inside your Lambda function, select the Code tab and then select the Upload a .ZIP file option, select your file, and upload it.

The testing functionality within Lambda isn’t extremely robust, but it’s better than nothing. Click on Actions and select Configure sample event. Paste the following code in there:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
{
"session": {
"new": false,
"sessionId": "session1234",
"attributes": {},
"user": {
"userId": null
},
"application": {
"applicationId": "amzn1.echo-sdk-ams.app.10549580-3661-4854-a3b4-49ac57e0d6f1"
}
},
"version": "1.0",
"request": {
"intent": {
"slots": {
"bus": {
"name": "bus",
"value": "304442"
}
},
"name": "GetNextBusIntent"
},
"type": "IntentRequest",
"requestId": "request5678"
}
}

This mimics what is sent to your function when a user invokes the intent on the Echo. You can use this sample event to test your code whenever you upload a change.

Checking the Logs

You shouldn’t run into any issues with the code we developed, but if you do now or with future Alexa skills you can check out the logs for your Lambda function. Under the Monitoring tab, there’s a link to view the logs inside of CloudWatch. I’ll warn you: the logging UI isn’t great. One example is that you need to be sure to order the logs by time occurred. For some reason this isn’t default inside of CloudWatch and it tripped me up a few times in the beginning.

Running Your Skill

Finally, let’s check out our skill on the Echo. Go back to the Amazon developer console that we were at earlier when we were adding our intent schema and sample utterances. If you saved those, you should be all set. Talk to your Echo and say: Alexa ask bus schedule for the next bus at stop 308638. And there you go!

You should get a response back saying how far the next bus is away. If you don’t, make sure your code matches what we went through above. Feel free to compare it to my finished code, too. There are a few differences, but it should largely be the same.

So what’s next? Try expanding on this–the MTA API allows you to ask for a bus stop by intersection. Add support for that. Or perhaps add a new intent. Maybe you can make this into a one-stop skill for NYC public transit and add subway and rail alerts.


Find this interesting? Follow me on Twitter and let’s discuss developing for the Amazon Echo and other side projects.

Project #3: Building a Database of Professional Athletes

For the third project, I decided to dig more into PhantomJS and, especially, CasperJS. PhantomJS is a “headless WebKit scriptable,” meaning essentially that it’s a browser without any of the displaying websites on the screen. CasperJS is a utility built on top of PhantomJS (and the Gecko headless browser SlimerJS) that gives you extra tools to make your job easier. I had done some minor scraping before, but nothing too intense.

What I set out to do was create a database of professional athletes in the United States. I started out with MLS, MLB, the NBA, and the NFL. The goal was to grab their name and photo and–if easily accessible–their team, position, and other information about them.

CasperJS !== Node

The first thing to keep in mind is that when working with CasperJS, you’re not working with Node. This might seem obvious until you decide you want to store scraped data directly into a database or pull in an NPM module like Underscore or Lodash. As the CasperJS docs will warn you: CasperJS can be installed via NPM for convenience only. Crossing over is difficult and probably not worth your time.

So then, what to do to bridge the gap? Depends what you’re trying to do. For the convenience methods that Underscore would have provided, I just got down into the vanilla JavaScript that Underscore abstracts away. And, really, there isn’t too much difference in readability between these two:

1
2
3
_.map(anchors, function(anchor){
return anchor.getAttribute('href');
});
1
2
3
Array.prototype.map.call(anchors, function(anchor){
return anchor.getAttribute('href');
});

What you lose from Underscore is edge case sensitivity and cross-browser support. These two, especially cross-browser support, isn’t really a concern for us as we always know in which environment our code will be executed and so we don’t need to worry about supporting older versions of IE or ancient Firefox browsers

For a situation where I wanted to save the data, I decided to add an extra step in between scraping and loading the data into the database where I would write the data as JSON to a file then load that file in a separate script execute by Node rather than CasperJS. Ideally it would be easy to connect via PhantomJS, but this extra step doesn’t require much in the way of extra data loading time. And since I’m using MongoDB, saving the data to JSON and loading it into the database is simple.

Crossing Over to the Other Side

Perhaps the part that took the most getting used to was the idea of different execution environments. You see, most of the code you write is in the PhantomJS environment–until you need to “reach” into the DOM of a given webpage. Then you enter the DOM environment. Confused? This graph helps clear it up a little:

Different PhantomJS Environments

The most important thing to note here is the fact that you are only returning native types from the DOM. This tripped me up a lot initially: you cannot return nodes from the DOM, so you have to gather up all of the data from the DOM before you return.

To execute code within the current page context, use the evaluate() method off of CasperJS. You can pass arguments through to the function you are evaluating, but–importantly–outside of those arguments the function doesn’t have access to any other variables or data from your PhantomJS environment.

Waiting for My Page to Load

Just like a normal browser, you need to be concerned with timeouts inside of PhantomJS/CasperJS as well. I ran into this situation while trying to scrape data from the NBA. Because of the fact that you are working with a headless browser and different execution environments, debugging this can be a little tricky.

What I found useful for this was CasperJS’s capture() method. This takes a screenshot of the current page whenever the method is invoked. This was useful for determining that the page was pulling in the list of players through AJAX and it wasn’t loading for the headless browser–showing a loading indicator instead. This allowed me to switch to a Node solution for the NBA as I could more quickly and easily gather player data from the NBA through JSON then I could through the page.

One piece of advice: unless you need images to be loaded, consider turning them off like so:

1
2
3
4
var casper   = require('casper').create({
verbose: true,
loadImages: false
});

This allows your scripts to run a lot faster and cuts down on the data that has to be transferred–easier on you and easier on the site you’re scraping, too.

Be Kind

One final note: scraping is legal, but don’t be a jerk. Don’t overload a given site with bot traffic and don’t reproduce the data you gather from the site. Doing either of these crosses over into the gray area of legality.

Interested in the code I developed for scraping pro athlete data? Check it out on Github.


Find this interesting? Follow me on Twitter and let’s discuss PhantomJS/CasperJS and other side projects.

CasperJS Error When Trying to Download -- Error: NETWORK_ERR: XMLHttpRequest Exception 101

While scraping a site, I ran into an issue with downloading images. I kept seeing this error:

1
Error: NETWORK_ERR: XMLHttpRequest Exception 101

And beyond that, the files were being created but they were completely empty. No date inside them at all.

Fixing this was simple. I changed this:

1
var casper  = require('casper').create({ verbose: true });

To this:

1
2
3
4
5
6
var casper  = require('casper').create({
verbose: true,
pageSettings: {
webSecurityEnabled: false
}
});

This permits the underlying Phantomjs to disable web security and allow cross-domain XHR. Be careful with this one: it’s called web security for a reason. Don’t use this if you’re passing through sensitive information.

Project #2: ToBuildSomething Blog

Sure enough, I went a bit out of order. I started this project of building more side projects without having a way to write it up. For the second week, I decided to rectify that.

I needed a static site generator and, ideally, I wanted it backed by JavaScript. I took a look at Hexo, Cabin, Blacksmith, Harp, Metalsmith, Middleman, and Jekyll. The last two are ruby-based but the rest are powered by JavaScript. I decided to go with Hexo because it was extensible without needing significant upfront configuration.

Hexo’s got what you need in a static site generator: easily generate a new blog $ hexo init, a new post $ hexo new title, and spin up a local server $ hexo server. It supports markdown, ejs inside of themes, and stylus for styles. There is a plugin system, so you can make it work how you want it to. It’s a nice platform and I recommend it.

Room for Improvement

With that said, there are some ways which Hexo could be improved. The documentation is good for simple actions, but there’s nothing to be found if you want to get a little more complex. I wrote about querying Hexo posts by category earlier and, best I can find, there’s no documentation for something like this.

Hexo’s also incredibly difficult to debug. Any error inside your template (and I ran into a handful, since spacebars is so similar to the way you reference Hexo helpers) gives you dozens of lines of red text in your terminal. Where’s the error? Hard to tell. There were some situations where I didn’t even get a line number.

Finally, I was unable to find a way to get the S3 deployment plugin to work. Due to Hexo’s lack of popularity (though it’s probably the most popular JavaScript-backed static site generator, it’s nowhere in the league of Jekyll or Middleman), there was no way to find help for that online either and it didn’t feel worth it to me to spend hours trying to debug it. Especially as the repository for the plugin didn’t accept issues.

All in all, I’m happy with Hexo. If you want a static site generator and want to use JavaScript, I recommend it. Be aware of the issues, but give it a shot.


Find this interesting? Follow me on Twitter and let’s discuss static site generators and other side projects.

Querying posts in Hexo

Hexo is a great platform if you want a static site generator (like Jekyll or Middleman), but you want it to be based on JavaScript. However, the documentation isn’t as complete as you’d hope.

One thing I wanted to do when I was setting up this blog was to have a widget in the sidebar that highlighted all of the posts about the projects themselves (rather than the posts talking about specific technical challenges I encountered). All of these posts have a category of project.

The syntax for this changed in Hexo 3.0, so [this post] didn’t work for me, despite being just about six months old. Finally I figured it out: you have to query off site.categories.

1
2
3
4
5
<% site.categories.findOne({name: 'project'}).posts.sort('date', -1).each(function(post){ %>
<li>
<a href="<%- url_for(post.path) %>"><%= post.title || '(no title)' %></a>
</li>
<% }) %>

Be sure you use findOne and not find, or else you’ll get an error.


Find this interesting? Follow me on Twitter and let’s discuss Hexo and side projects.