I’ve been writing a lot of JavaScript lately and I want a quick way to create a new JS file, especially since I’ve found that I follow the same steps every time: create the JS file, then add "use strict"; to the top of the file.
Luckily, fish shell makes it easy to create functions to streamline repetitive tasks.
1
function strict -a path
echo echo '"use strict";' > $path
end
The -a flag tells the function to accept an argument named path. Simple script to create a JavaScript file with "use strict"; at the top.
Find this interesting? Follow me on Twitter and let’s discuss alternative shells and side projects.
At the end of 2014, Amazon announced a new product–a voice-powered and wireless-enabled speaker called the Echo–that seemed a little strange for them. Amazon had ventured into hardware before with the Kindle, the Fire, and the ill-fated Fire Phone but the Echo didn’t seem to fit into Amazon’s goal of cheap hardware to enable people to buy more stuff. Afterall, initially the only thing you could buy through the Echo was music.
Amazon kicked off the Echo with invitation-only purchasing for Prime members, plus a $100 discount (bringing the price to $99) and a lengthy waitlist even after you purchased. Indeed, I knew people who purchased an Echo in January only to not receive it until May.
At first the Echo was a nice speaker, especially for the price, but nothing to really write home about. And it didn’t take advantage of the cloud-enabled features to the fullest extent. Then in late spring Amazon announced quietly that they would open up the Echo to developers to build upon.
The Alexa Skills Kit is a set of SDKs that enable developers to build voice-powered “skills” for the Echo. This move:
Gives developers SDKs in Java or JavaScript
Allows developers to host their own endpoints that the Echo would speak to or leverage the AWS Lambda compute service
Provides a structure for easily tapping into voice commands
The Project
For this project, I built a skill that probably doesn’t have wide appeal but is nonetheless one that I need. This skill allows the user to specify a New York City bus stop by its unique stop number (e.g. 308638) and hear spoken back how far the next bus is away. I always try to leave my apartment when the bus is 0.6 miles away from the stop so this will help me know when I should start packing up.
I’m disappointed to say, but developing for the Echo using Lambda was a total pain. If you need more than one file (and of course you do, because you’re not building a trivial skill, and any way, using the JavaScript SDK puts you at two files at a minimum) you have to compress and upload your files every time you make a change. The user interface leaves a lot to be desired, a common problem with AWS. Logging isn’t very robust or usable though it is built in, so that’s a plus.
On the other hand, being able to direct what Echo’s voice-avatar Alexa speaks back to you is fun. And thinking about how the user is going to interact with your skill is an interesting user experience challenge that most developers rarely get a chance to consider.
In this project, I’m going to walk you through the Amazon Alexa Skills Kit JavaScript SDK. Completely unfamiliar with the Alexa Skills Kit, Amazon Echo, and coding for them? Check out my tutorial for developing an Amazon Echo skill.
This is our constructor. Most of the rest of the SDK is adding methods onto this object’s prototype. We’re going to extend this constructor when we create our skill. It accepts an appId argument that you get when you create an Alexa skill inside the Amazon Developer Console.
SessionEndedRequest: function (event, context) { this.eventHandlers.onSessionEnded(event.request, event.session); context.succeed(); } };
The first two methods are really just syntactic sugar for two different event handlers. The third is invoked when the session ends and signals that the request has succeeded. You won’t really find yourself using these methods directly–these are primarily used inside the execute method we’ll look at later.
onSessionEnded: function (sessionEndedRequest, session) { } };
We’ve got four methods. Let’s examine them one-by-one.
onSessionStarted(sessionStartedRequest, session)
This is called when the session starts. It’s optional and used for any set up your skill needs.
onLaunch(launchRequest, session, response)
This is the only method that you’re required to override. This method is invoked when a user launches a skill without specifying an intent. Generally what you’ll want to do here is prompt the user to specify their intent.
onIntent(intentRequest, session, response)
When a user launches your skill and specifies what they want. You will most likely not override this method. The first argument is an object that has information about the request. We grab the intent off this request and look to the intentHandlers object to see if we have a handler for the given intent. If we do, we call the handler, otherwise we throw an error. We’ll look at the intentHandlers object in a bit.
onSessionEnded(sessionEndedRequest, session)
This is called when the session ends. Just like the onSessionStarted method, overriding this method is optional.
1
AlexaSkill.prototype.intentHandlers = {};
This is an object that will hold your intent handlers. These will be the same intents that you define in your intent schema inside the Amazon Developer’s Console.
The real meaty part of the code. This method is only invoked in the exported handler inside of your code in AWS Lambda, like so:
1 2 3 4
exports.handler = function(event, context) { var skill = new BusSchedule(); skill.execute(event, context); };
The first conditional checks that the applicationId on the session matches the appId we set in constructor.
The second conditional ensures that the session attributes is an object. And the third conditional invokes the onSessionStarted event handler if the session is marked as new.
Finally, we grab the request handler for the request’s type and calls the handler.
Of course, if at any point the code throws an error, the error will be logged and the fail method is invoked on the context object.
The Response Object
1 2 3 4
var Response = function (context, session) { this._context = context; this._session = session; };
Here’s another constructor, this time for the response, which accepts a context object and a session object. You’ll never instantiate with the response object directly–it’s passed through to the request handlers inside the execute method.
This object returns four methods: tell, tellWithCard, ask, and askWithCard.
We have two methods here which will respond to the user and end the session. First is tell, which accepts a string that Alexa will speak to the user, and tellWithCard, which accepts a string that Alexa speaks to the user, a string that serves as the card title, and a string that serves as the body of the card. The card is displayed within the Amazon Echo app.
These methods are just like the tell methods, except for two key differences. First, the session is kept open, waiting for a further response from the user. Second, the second argument is a string that Alexa will speak to the user if they haven’t responded to specify what they want.
The Response Object
These methods all assemble an object. Here’s a fully assembled object, with notes for each property:
{ version: '1.0', response: { outputSpeech: { // Always part of the response type: 'PlainText', // Always this value text: 'Example speech'// Text that will be spoken back to user }, shouldEndSession: true, // Always part of the response--boolean for if session has ended (no more user input) reprompt: { // Optional, if you want to customize what Alexa says if user doesn't respond correctly/at all outputSpeech: { type: 'PlainText', text: 'Example reprompt' } }, card: { // Optional, if you want to show a card in the Echo app type: 'Simple', title: 'Example title', content: 'Example card content' } }, sessionAttributes: {} // If you need some attributes to be kept in the session }
Note that you do not have to assemble this object yourself–the response methods will do it for you.
This is a look at the Amazon Alexa Skills Kit JavaScript SDK. It packs a lot of functionality into less than 180 lines of code. If you want to check out the code more in-depth check it out here.
Find this interesting? Follow me on Twitter and let’s discuss developing for the Amazon Echo and other side projects.
In this tutorial I’m going to walk you through how to create a “skill” for the Amazon Echo. The Echo is a voice-powered and wireless-enabled speaker from Amazon that was released in late 2014. A skill can be thought of as an app that increases the capabilities a user can take advantage of with the Echo.
The skill that we’re going to be building is a way for users to ask Alexa, the voice avatar behind the Echo, when the next bus will arrive at a New York City bus stop. To keep this simple and to focus on the Echo side of the equation, we’re going to require users to know the stop ID that they want. This means that they will have to request “stop three zero eight six three eight” and not “Myrtle at Flushing.”
Get a Key from the MTA
The first thing we need to do is request a developer key from the MTA. This is required for our API calls and can rarely (but sometimes) take up to thirty minutes to get a key, so let’s start off the process.
First go here and following the link to register for a key. The website says your application needs to be reviewed and that it will take up to thirty minutes to get your key. That hasn’t been my situation, but be aware. If it does take a little bit of time for you, we can get started while you wait.
Setup a Skill
From here we want to create a Lambda function. Lambda allows our first million invokations a month to be free and Amazon requires any code we would use on our own server to be behind https, so Lambda is the ideal approach for building an Alexa skill.
Log in to AWS and select Lambda. Make sure that you’re in the N. Virginia region, as Alexa Skills are currently only supported there. On the next page, click on the big button that says “Create a Lambda function.”
Amazon provides you with a number of blueprints that you can use to build your Lambda function and, since we’re going to be building an Alexa skill, select alexa-skills-kit-color-expert.
The event source type should have pre-selected Alexa Skills Kit. If not, select that and move on the next section. Give your function a name and a description (this is just for you, so name it something memorable). Leave the code that’s in there. We’ll come back and replace it later.
The handler we want to keep as index.handler. What this means is that Lambda is going to look for an exported function named handler inside of a file named index.js.
For the execution role, select Basic execution role. In the window that pops up, just go ahead and click “Allow.” You can then leave the memory and timeout settings the same before moving on to the next screen, reviewing everything and clicking on “Create function.”
On the next page, you should see on the right-hand side a string that starts off with “arn:” and ends with your function name. Note this, because we’ll need it in our next step.
The next thing we want to do is set up our Alexa Skill. Start by going to the Alexa Console and logging in. On the next page, select Alexa Skills Kit.
On the next page, select “Add a New Skill.” This will take you to a page where you can name your skill (which will be shown in the Echo app), create your invocation name (what users will say to start your skill), and the endpoint that we got when we created our Lambda function.
We’re going to name our skill Bus Schedule and use bus schedule for our invocation name. Add your Lambda function endpoint and submit the form.
The next screen asks us for our intent schema and sample utterances. The intent schema is JSON that defines our intents (the actions that user can take within our skill) and slots (our arguments within a given intent). The sample utterances are example invokations of our skill. It provides the Echo backend a guide to how the user will interact with our skill.
This defines two intents. The first is the GetNextBusIntent with a type of NUMBER. The other types available are DATE, TIME, DURATION, and LITERAL. The first three are straight-forward and LITERAL is just everything else. The second intent has no slots and will be available for users to ask for help on our skill.
GetNextBusIntent get next bus for {three zero four four four two|bus} GetNextBusIntentgetnextscheduledbusfor {three zero four four fiv six|bus} GetNextBusIntentgetnextarrivalfor {five zero eight nine zero five|bus} GetNextBusIntentgetbusfor {two zero eight nine seven nine|bus} GetNextBusIntentnextbusfor {three zero four four four two|bus} GetNextBusIntentnextscheduledbusfor {three zero four four fiv six|bus} GetNextBusIntentnextarrivalfor {five zero eight nine zero five|bus} GetNextBusIntentbusfor {two zero eight nine seven nine|bus} GetNextBusIntentgetnextbusat {three zero four four four two|bus} GetNextBusIntentgetnextscheduledbusat {three zero four four fiv six|bus} GetNextBusIntentgetnextarrivalat {five zero eight nine zero five|bus} GetNextBusIntentgetbusat {two zero eight nine seven nine|bus} GetNextBusIntentnextbusat {three zero four four four two|bus} GetNextBusIntentnextscheduledbusat {three zero four four fiv six|bus} GetNextBusIntentnextarrivalat {five zero eight nine zero five|bus} GetNextBusIntentbusat {two zero eight nine seven nine|bus} GetNextBusIntentgetnextbus {three zero four four four two|bus} GetNextBusIntentgetnextscheduledbus {three zero four four fiv six|bus} GetNextBusIntentgetnextarrival {five zero eight nine zero five|bus} GetNextBusIntentgetbus {two zero eight nine seven nine|bus} GetNextBusIntentnextbus {three zero four four four two|bus} GetNextBusIntentnextscheduledbus {three zero four four fiv six|bus} GetNextBusIntentnextarrival {five zero eight nine zero five|bus} GetNextBusIntentbus {two zero eight nine seven nine|bus} GetNextBusIntentfindnextbusat {three zero four four four two|bus} GetNextBusIntentfindnextscheduledbusat {three zero four four fiv six|bus} GetNextBusIntentfindnextarrivalat {five zero eight nine zero five|bus} GetNextBusIntentfindbusat {two zero eight nine seven nine|bus} GetNextBusIntentgetnextbusfor {five|bus} GetNextBusIntentgetnextscheduledbusfor {six eight|bus} GetNextBusIntentgetnextarrivalfor {seven nine five|bus} GetNextBusIntentgetbusfor {nine zero nine|bus} GetNextBusIntentnextbusfor {five|bus} GetNextBusIntentnextscheduledbusfor {six eight|bus} GetNextBusIntentnextarrivalfor {seven nine five|bus} GetNextBusIntentbusfor {nine zero nine|bus} GetNextBusIntentgetnextbusat {five|bus} GetNextBusIntentgetnextscheduledbusat {six eight|bus} GetNextBusIntentgetnextarrivalat {seven nine five|bus} GetNextBusIntentgetbusat {nine zero nine|bus} GetNextBusIntentnextbusat {five|bus} GetNextBusIntentnextscheduledbusat {six eight|bus} GetNextBusIntentnextarrivalat {seven nine five|bus} GetNextBusIntentbusat {nine zero nine|bus} GetNextBusIntentgetnextbus {five|bus} GetNextBusIntentgetnextscheduledbus {six eight|bus} GetNextBusIntentgetnextarrival {seven nine five|bus} GetNextBusIntentgetbus {nine zero nine|bus} GetNextBusIntentnextbus {five|bus} GetNextBusIntentnextscheduledbus {six eight|bus} GetNextBusIntentnextarrival {seven nine five|bus} GetNextBusIntentbus {nine zero nine|bus} GetNextBusIntentfindnextbusat {five|bus} GetNextBusIntentfindnextscheduledbusat {six eight|bus} GetNextBusIntentfindnextarrivalat {seven nine five|bus} GetNextBusIntentfindbusat {nine zero nine|bus}
GetNextBusIntent get next bus {threezerofourfourfourtwo|bus}
This roughly is equivalent to IntentName + spoken statement + slot. The slot can be anywhere inside the spoken statement but it must be in the curly braces with an example of what a user might request first, followed by a pipe, and then the slot name. Here, for example, is a made-up intent that has multiple slots spread out amongst the statement:
1
SetPropertyIntent give {Jane|owner} theproperty {Park Place|property}
Go ahead and save that–we’ll be coming back later, but for now we need to code up the skill.
Developing the Skill
Let’s first create a couple directories: src and speechAssets. Inside the speechAssets directory, create IntentSchema.json and SampleUtterances.txt. This step is optional, but I like storing a record of the intent schema and the sample utterances alongside the rest of the code. Paste the JSON we created earlier inside IntentSchema.json and the sample utterances inside SampleUtterances.txt.
Inside the src directory, create two files: AlexaSkill.js and index.js. The first file will be for the Alexa Skills Kit SDK, which you can grab here. The index.js file is going to be the main entry point for our skill.
At the top of our file, add the following variable declarations:
In this section we are requiring the Node package http so we can interact with the MTA API, the Alexa Skills Kit SDK, and specifying our Alexa skill’s app ID plus our MTA key. You can grab the app ID from the Skill Information section where you created your Alexa skill. The MTA_KEY is what you received when signing up with the MTA earlier.
Let’s next create a function that will assemble the endpoint we’ll communicate with to get the bus information from the MTA. It should be pretty self-explanatory:
This function accepts two arguments: the stopId the user has requested and a callback to handle the data once we’re finished grabbing it.
If you’re not familiar with Node’s http library, we’re listening to the incoming data stream. As soon as it’s finished we parse the string into JSON and pass it to our callback. If there’s an error, we log the error–we’ll look at how to view the logs later.
The next function we create is what will be invoked when the user invokes the GetNextBusIntent, i.e. when they request a bus stop. This function will invoke the getJsonFromMta function we just created and grab just the string we need from the response and send it back to the user.
var handleNextBusRequest = function(intent, session, response){ getJsonFromMta(intent.slots.bus.value, function(data){ if(data.Siri.ServiceDelivery.StopMonitoringDelivery[0].MonitoredStopVisit){ var text = data .Siri .ServiceDelivery .StopMonitoringDelivery[0] .MonitoredStopVisit[0] .MonitoredVehicleJourney .MonitoredCall .Extensions .Distances .PresentableDistance; var cardText = 'The next bus is: ' + text; } else { var text = 'That bus stop does not exist.' var cardText = text; }
var heading = 'Next bus for stop: ' + intent.slots.bus.value; response.tellWithCard(text, heading, cardText); }); };
A few things warrant further highlighting here.
The first is intent.slots.bus.value. We are reaching a few levels deep into the intent object and grabbing the bus slot. We know to grab the bus slot because that’s what we named it in our IntentSchema we created earlier.
The next thing we want to do is grab the data that was returned to the callback and traverse the structure. Unfortunately the MTA structure isn’t the most ideal, we need to go nine levels deep before we get what we’re looking for: the PresentableDistance which equates to something like 1.8 miles away, 2 stops away, or Arriving.
Finally, we call the tellWithCard method on the response object. If you’re interested in finding out more about the response object, check out my guide to the Alexa Skills Kit JavaScript SDK. But in this situation, just know that tellWithCard will tell the user the answer (via Alexa) and display a card in the Echo App. The first argument is what Alexa will say, the second argument is the header for the card, and the remaining argument is the body of the card.
Creating Our BusSchedule Constructor
Now what we’re going to do is extend the AlexaSkill SDK and create our own BusSchedule constructor. Again, if you want to dive deep you can check out the SDK guide, but you’ll be able to follow along without reading it first.
Add this code at the bottom of index.js:
1 2 3 4 5 6
var BusSchedule = function(){ AlexaSkill.call(this, APP_ID); };
What we’re doing here is creating our BusSchedule constructor and setting its prototype to the prototype from AlexaSkill. This means we get everything that AlexaSkill has and can extend or override as we see fit.
Event Handlers
In fact, we’ll override a method right away. There are four event handlers we can override: onSessionStarted, onLaunch, onIntent, and onSessionEnded. We are only required to override onLaunch. You’re unlikely to override onIntent and onSessionStarted and onSessionEnded are available for setup and tear down.
onLaunch is the event handler that is called when a user has launched the skill but hasn’t specified which intent they want. For example, Alexa ask bus schedule versus Alexa ask bus schedule for the next bus at stop 304442.
1 2 3 4 5 6 7 8
BusSchedule.prototype.eventHandlers.onLaunch = function(launchRequest, session, response){ var output = 'Welcome to Bus Schedule. ' + 'Say the number of a bus stop to get how far the next bus is away.';
var reprompt = 'Which bus stop do you want to find more about?';
response.ask(output, reprompt); };
All event handlers accept arguments of a request, a session, and a response. Ours is pretty simple–we’re just telling the user what they can do with our skill–but you can make it more complext if you’d like. We’re using the ask method on the response object, which will keep the connection open and listen for a user reply. The reprompt is optional, but allows us to poke a user if they don’t respond soon enough to Alexa’s questioning.
Intent Handlers
The SDK defines an intent handlers object, but leaves it up to you to define the handlers themselves. These correspond to the intents that we specified in our intent schema that we created earlier. In our case, GetNextBusIntent and HelpIntent. Add this to your index.js:
HelpIntent: function(intent, session, response){ var speechOutput = 'Get the distance from arrival for any NYC bus stop ID. ' + 'Which bus stop would you like?'; response.ask(speechOutput); } };
Our first intent handler just invokes our handleNextBusRequest function we created earlier, while our second intent handler tells the user more about the skill we’ve created then keeps the connection open to allow the user to specify their next move.
Finally, we tie all the code together and export our handler:
1 2 3 4
exports.handler = function(event, context) { var skill = new BusSchedule(); skill.execute(event, context); };
Uploading Our Code to AWS Lambda
Now we want to put our code up on AWS Lambda. We first need to compress our index.js and AlexaSkill.js files, but make sure you are compressing just the files and not the folder that contains the files. This is really important, otherwise Lambda won’t be able to find your code.
Inside your Lambda function, select the Code tab and then select the Upload a .ZIP file option, select your file, and upload it.
The testing functionality within Lambda isn’t extremely robust, but it’s better than nothing. Click on Actions and select Configure sample event. Paste the following code in there:
This mimics what is sent to your function when a user invokes the intent on the Echo. You can use this sample event to test your code whenever you upload a change.
Checking the Logs
You shouldn’t run into any issues with the code we developed, but if you do now or with future Alexa skills you can check out the logs for your Lambda function. Under the Monitoring tab, there’s a link to view the logs inside of CloudWatch. I’ll warn you: the logging UI isn’t great. One example is that you need to be sure to order the logs by time occurred. For some reason this isn’t default inside of CloudWatch and it tripped me up a few times in the beginning.
Running Your Skill
Finally, let’s check out our skill on the Echo. Go back to the Amazon developer console that we were at earlier when we were adding our intent schema and sample utterances. If you saved those, you should be all set. Talk to your Echo and say: Alexa ask bus schedule for the next bus at stop 308638. And there you go!
You should get a response back saying how far the next bus is away. If you don’t, make sure your code matches what we went through above. Feel free to compare it to my finished code, too. There are a few differences, but it should largely be the same.
So what’s next? Try expanding on this–the MTA API allows you to ask for a bus stop by intersection. Add support for that. Or perhaps add a new intent. Maybe you can make this into a one-stop skill for NYC public transit and add subway and rail alerts.
Find this interesting? Follow me on Twitter and let’s discuss developing for the Amazon Echo and other side projects.
For the third project, I decided to dig more into PhantomJS and, especially, CasperJS. PhantomJS is a “headless WebKit scriptable,” meaning essentially that it’s a browser without any of the displaying websites on the screen. CasperJS is a utility built on top of PhantomJS (and the Gecko headless browser SlimerJS) that gives you extra tools to make your job easier. I had done some minor scraping before, but nothing too intense.
What I set out to do was create a database of professional athletes in the United States. I started out with MLS, MLB, the NBA, and the NFL. The goal was to grab their name and photo and–if easily accessible–their team, position, and other information about them.
CasperJS !== Node
The first thing to keep in mind is that when working with CasperJS, you’re not working with Node. This might seem obvious until you decide you want to store scraped data directly into a database or pull in an NPM module like Underscore or Lodash. As the CasperJS docs will warn you: CasperJS can be installed via NPM for convenience only. Crossing over is difficult and probably not worth your time.
So then, what to do to bridge the gap? Depends what you’re trying to do. For the convenience methods that Underscore would have provided, I just got down into the vanilla JavaScript that Underscore abstracts away. And, really, there isn’t too much difference in readability between these two:
What you lose from Underscore is edge case sensitivity and cross-browser support. These two, especially cross-browser support, isn’t really a concern for us as we always know in which environment our code will be executed and so we don’t need to worry about supporting older versions of IE or ancient Firefox browsers
For a situation where I wanted to save the data, I decided to add an extra step in between scraping and loading the data into the database where I would write the data as JSON to a file then load that file in a separate script execute by Node rather than CasperJS. Ideally it would be easy to connect via PhantomJS, but this extra step doesn’t require much in the way of extra data loading time. And since I’m using MongoDB, saving the data to JSON and loading it into the database is simple.
Crossing Over to the Other Side
Perhaps the part that took the most getting used to was the idea of different execution environments. You see, most of the code you write is in the PhantomJS environment–until you need to “reach” into the DOM of a given webpage. Then you enter the DOM environment. Confused? This graph helps clear it up a little:
The most important thing to note here is the fact that you are only returning native types from the DOM. This tripped me up a lot initially: you cannot return nodes from the DOM, so you have to gather up all of the data from the DOM before you return.
To execute code within the current page context, use the evaluate() method off of CasperJS. You can pass arguments through to the function you are evaluating, but–importantly–outside of those arguments the function doesn’t have access to any other variables or data from your PhantomJS environment.
Waiting for My Page to Load
Just like a normal browser, you need to be concerned with timeouts inside of PhantomJS/CasperJS as well. I ran into this situation while trying to scrape data from the NBA. Because of the fact that you are working with a headless browser and different execution environments, debugging this can be a little tricky.
What I found useful for this was CasperJS’s capture() method. This takes a screenshot of the current page whenever the method is invoked. This was useful for determining that the page was pulling in the list of players through AJAX and it wasn’t loading for the headless browser–showing a loading indicator instead. This allowed me to switch to a Node solution for the NBA as I could more quickly and easily gather player data from the NBA through JSON then I could through the page.
One piece of advice: unless you need images to be loaded, consider turning them off like so:
1 2 3 4
var casper = require('casper').create({ verbose: true, loadImages: false });
This allows your scripts to run a lot faster and cuts down on the data that has to be transferred–easier on you and easier on the site you’re scraping, too.
Be Kind
One final note: scraping is legal, but don’t be a jerk. Don’t overload a given site with bot traffic and don’t reproduce the data you gather from the site. Doing either of these crosses over into the gray area of legality.
This permits the underlying Phantomjs to disable web security and allow cross-domain XHR. Be careful with this one: it’s called web security for a reason. Don’t use this if you’re passing through sensitive information.
Sure enough, I went a bit out of order. I started this project of building more side projects without having a way to write it up. For the second week, I decided to rectify that.
I needed a static site generator and, ideally, I wanted it backed by JavaScript. I took a look at Hexo, Cabin, Blacksmith, Harp, Metalsmith, Middleman, and Jekyll. The last two are ruby-based but the rest are powered by JavaScript. I decided to go with Hexo because it was extensible without needing significant upfront configuration.
Hexo’s got what you need in a static site generator: easily generate a new blog $ hexo init, a new post $ hexo new title, and spin up a local server $ hexo server. It supports markdown, ejs inside of themes, and stylus for styles. There is a plugin system, so you can make it work how you want it to. It’s a nice platform and I recommend it.
Room for Improvement
With that said, there are some ways which Hexo could be improved. The documentation is good for simple actions, but there’s nothing to be found if you want to get a little more complex. I wrote about querying Hexo posts by category earlier and, best I can find, there’s no documentation for something like this.
Hexo’s also incredibly difficult to debug. Any error inside your template (and I ran into a handful, since spacebars is so similar to the way you reference Hexo helpers) gives you dozens of lines of red text in your terminal. Where’s the error? Hard to tell. There were some situations where I didn’t even get a line number.
Finally, I was unable to find a way to get the S3 deployment plugin to work. Due to Hexo’s lack of popularity (though it’s probably the most popular JavaScript-backed static site generator, it’s nowhere in the league of Jekyll or Middleman), there was no way to find help for that online either and it didn’t feel worth it to me to spend hours trying to debug it. Especially as the repository for the plugin didn’t accept issues.
All in all, I’m happy with Hexo. If you want a static site generator and want to use JavaScript, I recommend it. Be aware of the issues, but give it a shot.
Find this interesting? Follow me on Twitter and let’s discuss static site generators and other side projects.
Hexo is a great platform if you want a static site generator (like Jekyll or Middleman), but you want it to be based on JavaScript. However, the documentation isn’t as complete as you’d hope.
One thing I wanted to do when I was setting up this blog was to have a widget in the sidebar that highlighted all of the posts about the projects themselves (rather than the posts talking about specific technical challenges I encountered). All of these posts have a category of project.
The syntax for this changed in Hexo 3.0, so [this post] didn’t work for me, despite being just about six months old. Finally I figured it out: you have to query off site.categories.
I’ve recently taken a liking to Meteor, which you can see with my first project, Classboard.
Meteor’s interesting for a lot of reasons, but one thing that stands out to me is its age. Meteor has reached its 1.0 release, so it is stable enough to consider seriously. On the other hand, it’s young enough that there’s not as many people building with or on it as there are for Rails, Django, or even Express.
One place where this is apparent is with Atmosphere, which is Meteor’s package manager. You can use NPM packages with Meteor, but it takes some finangling. The Meteor package manger is comparatively sparse, so if you have an idea for a package there’s a chance someone hasn’t already gotten to it.
I’ve never written an NPM package before, but I do have experience writing ruby gems (here’s one). I have to say–ruby gems aren’t difficult, but Meteor packages are a breeze.
I decided to start off with a simple package that was a step beyond “Hello World” but not to the level of Meteor Toys. What I settled on was a lorem ipsum generator that allows you to have stock ipsum (lorem ipsum dolor amit and so on…) or to seed the generator with your own text. You can also optionally specify how many words you want and if you wanted your text returned to you nested inside of an HTML tag or on its own.
Set Up Your Package Structure
The first thing you want to do is set up your package structure and Meteor makes this easy for you. First make sure you’re in a Meteor app and you have a directory called packages. Then, in the command line, run:
Replacing the username with your Meteor account username and the package name with what you’ll be calling the package. I called mine Lorem, so the rest of this post will be using that as the package name.
This command will create a directory named the same as your package (lorem) and four files: README.md, lorem-tests.js, lorem.js, and package.js.
The file package.js is where your configuration happens. In that file you’ve got three methods that allow you to do your configuration: Package.describe, Package.onUse, and Package.onTest. The first takes an object of configuration details, while the next two each take a function. Let’s go over each one-by-one.
describe
1 2 3 4 5 6 7 8 9 10 11
Package.describe({ name: 'dcoates:lorem', version: '1.0.0', // Brief, one-line summary of the package. summary: 'A lorem ipsum generator. Use straight ipsum or customize with your own words.', // URL to the Git repository containing the source code for this package. git: 'https://github.com/dustincoates/lorem', // By default, Meteor will default to using README.md for documentation. // To avoid submitting documentation, set this field to null. documentation: 'README.md' });
This method is what Atmosphere will hook into for displaying information about your package. This should all be pretty self-explanatory. The name is technically optional, as the default behavior is for the package name to come from the directory holding it. Summary and version are only required if you plan on publishing the package.
The onUse method accepts an argument of api and allows for configuration inside the method body.
versionsFrom
The first configuration we see here is versionsFrom, where you can specify which versions of core packages (such as jQuery) should be used. This accepts either a string or an array of strings referring to versions.
The use method accepts multiple arguments. The first one is either a string or an array of strings that correspond to packages that your package depends on. If you are using non-core packages, you must also specify the version of that package (e.g. 'accounts@1.0.0'). You can leave this off for core packages.
The second argument refers to the architecture. Is your package only available on the server or on the client? You have as options here 'server', 'client', 'web.browser', and 'web.cordova'.
You also have the options to weakly depend on a package. When set to true it means that you do not force the package that is a dependency to load, but can use it if it’s loaded by another package or by the user.
You also have the unordered option, which means that the dependency can load after your package. The default behavior is that all dependencies load before the package. The official Meteor docs say that this can help fix circular dependency issues.
addFiles
1
api.use(['templating'], 'client');
The next configuration is addFiles. These are the files that make up your package. This accepts up to two arguments. The first one is a string or array of strings specifying your source files. The second, optional, argument allows you to only export these files on the server or the client. This accepts the same strings as the architecture arguments as use above.
export
1
api.export('Lorem');
These are the variables you want to be made available with your package. The first argument is a string representing the variable name, while the second optional argument is–again–the architecture where you want this to be available.
imply
I didn’t use this option, but it allows you to give users of your package access to other packages. You can specify a string (the name of the package you want users to be able to access) or an array of such strings.
This is just like the onUse function but is specifically for our unit tests. Here we can explicitly use our package because we’ve already configured the individual files inside the onUse function.
Meteor uses TinyTest for package unit testing. While not as full featured as something like Jasmine, it gets the job done. The documentation for this is, well, non-existent. Thankfully for us someone updated the readme less than two weeks ago, so we don’t have to go digging around the source code as much. The readme lets us know that we have the following methods at our disposal:
test.ok(doc)
test.expect_fail()
test.fail(doc)
test.exception(exception)
test.runId()
test.equal(actual, expected, message, not)
test.notEqual(actual, expected, message)
test.instanceOf(obj, klass, message)
test.notInstanceOf(obj, klass, message)
test.matches(actual, regexp, message)
test.notMatches(actual, regexp, message)
test.throws(func)
test.isTrue(v, msg)
test.isFalse(v, msg)
test.isNull(v. msg)
test.isNotNull(v, msg)
test.isUndefined(v, msg)
test.isNotUndefined(v, msg)
test.isNaN(v, msg)
test.isNotNaN(v, msg)
test.include(s, v, message, not)
test.notInclude(s, v, message)
test.length(obj, expected_length, msg)
Use those as you wish.
Beyond the lack of documentation, TinyTest is really easy to pick up. Here’s an example test from the Lorem package:
Run your tests with the meteor test-packages command:
1
$ meteor test-packages <username>:<package_name>
NPM.depends
There is a fourth method that we don’t see normally, but you need if you are utilizing NPM packages: Npm.depends(). This also takes an object where the keys are for the names of NPM packages of which you’re using and keys for the associated versions.
Creating the Package
From here, creating the package is just like creating code in any other part of your app. You can create templates inside of an html document and JavaScript that can be accessed by apps that use your package.
Very important to note: if you specified an export in your package.js file, it must be declared without the var keyword. This instantiates it as a global variable within the package.
Publishing the Package
When you’re finished with your package, you can share it with the world on Atmosphere. Like usual, Meteor has made this simple for us. Go into the directory that hosts your package and run the following command:
1
$ meteor publish --create
It may take a while to see it, but if you go to atmospherejs.com// you should see your package up there for everyone to use.
I teach classes once to three times a month at General Assembly on general programming topics. So far in my classes I’ve been sharing resources, URLs, and code snippets through Google Docs which the students can access via a bit.ly URL. This has worked out well so far, but it’s an extra 15 minutes before class of setting it up and it isn’t really built for this purpose.
For my first project I decided to build a replacement. These were the general requirements:
Updates needed to be pushed to the client immediately–no browser refresh
It had to support inclusion of URLs, code snippets, and plain text
URLs had to be autolinked
Code snippets had to be syntax highlighted
The creator of the board is the only one who can push updates, so there had to be user accounts
These were some things I would have liked to have, time permitting:
People accessing the board would get a notification when a new update was posted
Files could be attached
PDFs could be embedded
Users could “subscribe” to different resource lists
Considering the first requirement (that updates need to be pushed to the client immediately), a Node based framework coupled with websockets made the most sense. I’d been having fun with Meteor and had started using it in my Programming for Non-programmers classes, so it made perfect sense here.
Collections
I ended up with two different collections: Boards and Resources. Boards roughly corresponded to a single workshop and it would contain all of the resources like code snippets.
Boards
Boards were as simple as you could get. To create a board, a user just clicked a “create board” link and that was it. There was no option to name the board at the current moment, though that might be something that can be added in the future. Because there was no user input, there was no need for validations and so the method for inserting a board was as follow:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Meteor.methods({ insertBoard: function(){ var user = Meteor.user(), board = { userId: user._id, username: user.username, dateCreated: newDate() }, boardId = Boards.insert(board);
return { _id: boardId }; } });
Resources
Resources were a bit more complex. Not only did they accept user input, but they also were of set types (url, code, and plain text to start). If I were doing this in Rails, I’d use an enum. But with mongo the benefits of using an enum aren’t as clear.
To get at this, I needed a method directly off of the Resources collection. I wrote about this in more detail in another post, but the simple gist is that I used Underscore’s extend method to add a new method to Resource:
Another requirement was the syntax highlighting of code snippets. I wrote about this in-depth, so I won’t recount all of it but the main takeaway is that I used Prism directly without using a package.
User Accounts
Integrating user accounts was as simple as always. Run the following command in the terminal:
1
$ meteor add accounts-password accounts-ui
Add {{> loginButtons}} to my view and there we go.
The frustrating thing, however, was styling the thing. Even switching out accounts-ui for accounts-ui-unstyled didn’t do a ton of good, because I was unable to easily change the structure of the underlying HTML. In the future I’ll likely create a Meteor package internal to my app to get around this.
Notifications
The next thing I wanted to add was browser notifications whenever a new update was pushed. I’d never tackled this before and expected it to be tricky. Thankfully the browser APIs have gotten to a point where adding notifications was incredibly simple. Plus, MDN’s documentation was fantastic. Here’s the code:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
var createNotification = function(message, options){ var notification = new Notification(message, options); setTimeout(notification.close.bind(notification), 6000); }
The first function, createNotification, is what actually creates the notification that will be shown to the user. We provide it a string for the message and an object for the options. The options can include a body and an icon.
In this notification “Emogotchi says” is the message, the text below it is the body, and the image to the right is the icon.
Next we close the notification after six seconds with the setTimeout function.
We don’t do it here, but we could ensure that users only see new notifications by listening for the show event on the notification and storing that on the user object. Then users only get notifications for updates sent after they last received a notification.
There are also events for error, click, and close. Note that the close notification is triggered whether your code closed the notification or if the user did.
The second function, sendNotification, is what’s going to be used in other parts of the app.
First we check to see if the environment we’re in (typically the browser) supports the notifications API. If it does, we see if the user has already granted us permission to send notifications. If not, we request permission. Requestion permission takes a callback that accepts an argument of the permission. Only if the user has granted permission do we go ahead and send the notification.
This was the last thing I added into the code and there were a few things I plan to improve. The first one was mentioned above: store on the user record when the last notification they received was. Right now what happens is when they come back all of the notifications come back to them.
The second thing I’d like to change is what happens if multiple updates go out in a short time span. Right now they pile up. This is okay if there are just two or three of them but any more and the students will likely get distracted trying to close them. That’s the last thing we need during class.
Thankfully there’s an easy way at this. By adding a tag value to the options object we pass to the notification, if there are multiple notifications with the same tag then only one will be shown at a time.
Work Left to Be Done
This side project particularly suffered from free time, as incidentally I had a three day class to teach. But I imagine that since each side project is just a week long there will be things that I want to improve or add. I’ve mentioned a couple things with the notifications, but I’d also like to improve the styling (the admin section particularly), add the ability to embed PDFs, and autogenerate a short URL anytime a new board is created.
Interested in these projects? Follow me on Twitter and let’s discuss meteor and side projects.
