Subclassing in JavaScript is quite useful. I used JavaScript subclasses to create this Star Wars inspired game.

You can play it right now, or check out the code on Github. The rest of this insanely long post will demonstrate how to use JavaScript subclassing to build the foundations of a similar game or visualization.

Subclassing overview

A class can be written to create fleets of similar objects with shared properties and methods. For example, if you're building the game Frogger, you can create a class called Vehicle that will produce a single instance of one of those pesky little cars. Let's imagine that your Vehicle class has one property (location, to track a car's current location) and one method (move, to drive a car forward). Based on the difficulty of the game level, you can use your Vehicle class to produce as many cars as you want to hinder that frog from its goal.

But what if you get the idea to introduce a new type of vehicle that nefariously changes lanes at random, or a friendly car that scoops up the froggie and brings him to the pond? You could certainly add those features to your original Vehicle class, but then all instances of Vehicle would have those new properties and methods, and it may prove difficult to track and turn them on for certain Vehicle instances based on your game design (e.g. produce one and only one friendly car per level).

Another option could be replicating the existing class for these new vehicle types. Now you'll have separate Vehicle, AggressiveVehicle, and FriendlyVehicle classes, and you can instantiate them all independently to your heart's delight. However, you've likely duplicated a lot of similar code in each of these classes - such as the location property and the move method. If you ever want to change how all the vehicles move, you'll now need to remember to change the code in each of the classes. There's got to be a better way to do this. And there is - subclassing!

Subclassing allows you to create a fleet of objects that look vaguely similar to other objects. Subclasses are able to "share" properties and methods from their "parent" or "superclass". In JavaScript, subclassing is implemented via delegation. Rather than keeping the Frogger metaphor going, let's transition to a galaxy far, far away...

Starfield simulation

Like all great games, SubclassWars.js is based on a Windows screensaver.

We want to simulate moving through space. A good first step could be randomly drawing a bunch of white dots on a black background. They don't even have to move - just get them to show up.

Let's create a base class called Star:

    /**
    *  Star class: the base class
    */
    var Star = function(x, y){
    // "Stars" are just HTML spans 
    this.$node = $('<span class="star"></span');
    // Location along x-axis
    this.x = x;
    // Location along y-axis                                   
    this.y = y;
    this.setPosition();
    };
    // // Position the HTML span based on x and y
    Star.prototype.setPosition = function(){
    var positionSettings = {
        top: this.x,                      
        left: this.y
    };
    var styleSettings = {
        "border": "2px",
        "border-style": "solid",
        "border-color": "white",
        "border-radius": "2px",
        "position": "absolute"
    };
    this.$node.css(styleSettings);
    this.$node.css(positionSettings);
    };

    /**
     *  Setup starfield size
     */
    var height = 200;                               
    var width = 700;

    /**
     *  Build star helper function
     */
    var buildStar = function() {                    
    var star = new Star((Math.random() * width),
                        (Math.random() * height)
                        );
    $('#stars').append(star.$node);
    };

    /**
    *  Initialize the game
    */
    $(document).ready(function() {

    $('.starfield').css({
        "height": height,                          
        "width": width,
        "background-color": "black",
        "position":"relative"
    });

    for (var i = 0; i < 20; i++) {
        buildStar();
    }

    });

Hello, stars! Note that I've used the pseudoclassical instantiation pattern (rather than the functional or prototypal patterns) to build the Star class, so new stars need to be instantiated with the "new" keyword. Another important detail for this and any other HTML visualizations is that the origin (the [0,0] location) of any HTML element or document is the upper left corner.

Now, let's make those stars twinkle to show you how subclassing works. We will need to refactor our base Star class slightly:

/**
 *  Star class: the base class
 */
 var Star = function(x, y, timeBetweenSteps){
   this.$node = $('<span class="star"></span'); 
   this.x = x;
   this.y = y;
   // New param to create a loop
   this.timeBetweenSteps = timeBetweenSteps;
   this.step();
   this.setPosition();
 };
 Star.prototype.step = function(){
   setTimeout(this.step.bind(this),
     this.timeBetweenSteps);
 };
 Star.prototype.setPosition = function(){
   var positionSettings = {
     top: this.y,                                
     left: this.x
 };
   var styleSettings = {
     "border": "2px",
     "border-style": "solid",
     "border-color": "white",
     "border-radius": "2px",
     "position": "absolute"
   };
   this.$node.css(styleSettings);
   this.$node.css(positionSettings);
 };
 
 /**
 *  TwinkleStar class (our first subclass!)
 */
 var TwinkleStar = function(x, y, timeBetweenSteps){
   Star.call(this, x, y, timeBetweenSteps);
 };
 TwinkleStar.prototype = Object.create(Star.prototype);
 TwinkleStar.prototype.constructor = TwinkleStar;
 TwinkleStar.prototype.step = function(){
   Star.prototype.step.call(this);
   this.$node.toggle();
 };

 /**
  *  Setup starfield size
  */
 var height = 200;                               
 var width = 700;
 
 /**
  *  Build star helper function
  */
 var buildStar = function() {
   var star = new Star((Math.random() * width),
                       (Math.random() * height),
                       100
                       );
   $('#stars').append(star.$node);
 };
 
 var buildTwinkleStar = function() {
   var star = new TwinkleStar((Math.random() * width),
                       (Math.random() * height),
                       100
                       );
   $('#twinkle').append(star.$node);
 };
 
 /**
 *  Initialize the game
 */
 $(document).ready(function() {
 
   $('.starfield').css({
     "height": height,                          
     "width": width,
     "background-color": "black",
     "position":"relative"
   });
   for (var i = 0; i < 20; i++) {
     buildStar();
     buildTwinkleStar();
   }
 
 });

Now we're getting somewhere!

A lot happened in the last in the last iteration of code, and it's important to highlight some things now. First, we introduced a new method to our base Star class called 'step' that essentially mimics a game loop with the native JS function setTimeout. This is a little hacky and definitely not what you want to use for a real game, but this is just a demo, and hey, it's me.

TODO: fix stars

With our new step method built into the base Star class, we can now create lots of different subclasses of Star that add unique functionality. TwinkleStar twinkles, but we might want other stars that don't twinkle. How about stars that move instead of twinkle? Good idea, but we're not ready to get there just yet.

Notice that we are calling setTimeout within the step method, and then passing in step as the function argument for setTimeout. setTimeout takes two arguments, a function to execute after a specified time, and that specified time. However, when setTimeout receives the function argument, it will simply accept the function value without maintaining any reference to the star instance that called step in the first place. That means that step will be called again as a free function invocation - 'this' will be bound to the global window object. That's not what we want at all. One solution is to set a context variable (var that = this) in the step function and use it as context for calling step. Another solution, which I've used, is to use bind to create a function called step where "this" is specifically bound to the star that called step. That was a mouthful, but an important mouthful.

Let's talk about how we created the TwinkleStar subclass. We created a function called TwinkleStar with the same parameter signature as Star. Within that function, the Star class function is called using the .call method to bind 'this' to the new TwinkleStar object being created. Now, the new TwinkleStar will be set up with the base properties of a regular ol' Star (e.g. x, y, timeBetweenSteps, $node). Outside of the TwinkleStar function, we set up property and method delegation to the Star class using the Object.create pattern so that our new TwinkleStars can use any of the methods from the Star class (e.g. step, setPosition). Then we changed the constructor property to the TwinkleStar class so that any new TwinkleStars will be able to state that they are an "instanceof" TwinkleStar (and not Star). Finally, the coolest thing we do is write a new step method for TwinkleStar where we specify our unique action for all TwinkleStars. First we call the original Star step method, and then, crucially, we use jQuery to toggle the star span node on and off each time step gets executed - AKA making them twinkle.

That's it! JavaScript subclassing in the pseudoclassical pattern.

The lost city of Z

Now it's time to get serious about our screensaver replica. We need to make moving stars. But not just any moving stars, we need our stars to move in such a way that it feels like we are being whisked away to a faraway planet for a new and exciting adventure. The stars need to move "towards" the screen, even though we can only place these HTML spans on a flat 2D surface. Yikes. Now, there are lots of really impressive implementations of moving starfields on the web, so it's definitely possible, but will require some math. After some searching, I discovered a very simple procedure that can calculate an x and y position for a star taking into account a simulated z-axis property.

I'll walk through our changes in the code. Note that I'm not including the code for the Star or TwinkleStar classes since those won't be changing anymore.

/**
 *  MoveStar class
 */
 var MoveStar = function(x, y, z, timeBetweenSteps){
   this.z = z;
   Star.call(this, x, y, timeBetweenSteps);
 };
 MoveStar.prototype = Object.create(Star.prototype);
 MoveStar.prototype.constructor = MoveStar;
 MoveStar.prototype.setPosition = function(){
 
   var k = 128.0 / this.z;
   // Translate into x-pos in 2D
   var px = (this.x * k) + (width / 2);
   // Translate into y-pos in 2D
   var py = (this.y * k) + (height / 2);
 
   var positionSettings = {
     top: py,                                
     left: px
   };
   
   // Stars "grow" larger when closer
   var size = ( (1 - (this.z / 32)) * 3 );
 
   var styleSettings = {
     "border": size,
     "border-style": "solid",
     "border-color": "white",
     "border-radius": "2px",
     "position": "absolute"
   };
   this.$node.css(styleSettings);
   this.$node.css(positionSettings);
 
 };
 MoveStar.prototype.step = function(){
 
   this.setPosition();
  
   // Each tick, decrement z slightly
   this.z -= 0.2;                              
 
   if ( this.z <= 0 ||     
         (this.y * 128.0 / this.z) + (height / 2) < 0 ||
         (this.y * 128.0 / this.z) + (height / 2) > height ||
         (this.x * 128.0 / this.z) + (width / 2) < 0 ||
         (this.x * 128.0 / this.z) + (width / 2) > width 
     ){                                 
    // When star "reaches" the screen, send it far away!!
    this.x = randomInRange(25);              
    this.y = randomInRange(25);
    this.z = 31;
   }
 
   Star.prototype.step.call(this);
 };
 /**
  *  Helper function
  */
 var randomInRange = function(n){
       var sign = Math.random() > 0.5 ? 1 : -1;
       return sign * Math.floor( Math.random() * n);
 };
 
 /**
  *  Setup starfield size
  */
 var height = 400;                               
 var width = 700;
 
 /**
  *  Build star helper function
  */
 var buildStar = function() {
   var star = new Star((Math.random() * width),
                       (Math.random() * height),
                       200
                       );
   $('#stars').append(star.$node);
 };
 
 var buildTwinkleStar = function() {
   var star = new TwinkleStar((Math.random() * width),
                       (Math.random() * height),
                       200
                       );
   $('#twinkle').append(star.$node);
 };
 var buildMoveStar = function() {
   var star = new MoveStar(randomInRange(50),
                       randomInRange(50),
                       Math.floor( 32 * Math.random() + 1),
                       50
                       );
   $('#moving').append(star.$node);
 };
 
 /**
 *  Initialize the game
 */
 $(document).ready(function() {
 
   $('.starfield').css({
     "height": height,                          
     "width": width,
     "background-color": "black",
     "position":"relative"
   });
   for (var i = 0; i < 20; i++) {
     buildStar();
     buildTwinkleStar();
   }
   for (var i = 0; i < 100; i++) {
     buildMoveStar();
   }
 
 });

The MoveStar subclass includes an additional parameter called z that will represent the star's position on the z-axis. A value of 32 is as "far away" as possible and a value of zero is as "close" as possible (basically at the screen). The MoveStar subclass will have to simulate motion along the z-axis by starting the stars quite far away, decrementing their z property with each "step" in the setTimeout loop, and then translating their current posision into an x and y position on the screen. Once the stars reach the screen, we reposition them again far away. Because the stars are being replaced in random positions around the center of the 'div' using the new randomInRange helper function, each star will "move" towards its respective quadrant over time as the decrementing z value exaggerates its respective x and y positions on screen. This is key to the visualization looking realistic. We've also decided to have the stars "grow" larger as they get closer to the screen, which further helps simulate motion.Through the magic of subclassing, the "step" and "setPosition" methods were both overwritten while still referencing the superclass methods.

The Tie Fighters That Bind

We've now recreated our 90's screensaver. This alone is a very cool accomplishment. But we're still not yet in game territory yet. Specifically, enemy territory. The next change I'll make is to introduce a new subclass of MoveStar called TieFighter. This should be easy.

/**
 *  TieFighter class
 */
 var TieFighter = function(x, y, z, timeBetweenSteps){
   MoveStar.apply(this, arguments);
   this.$node = $('<img src=".subclass-wars/tiefighter.png"></img>');
   this.step();
 };
 TieFighter.prototype = Object.create(MoveStar.prototype);
 TieFighter.prototype.constructor = TieFighter;
 TieFighter.prototype.step = function() {
   var size = ( (1 - (this.z / 32)) * 100 );
   this.$node.css({height: size, width: size});
   MoveStar.prototype.step.call(this);
 };

Okay, that worked, sort of. Simply by replacing the type of DOM element with an img tag, we've turned our MoveStars into Tiefighters! But there's clearly some kind of CSS border situation going on that we'll have to investigate. And the TieFighters are not disappearing immediately when they hit the bottom or right side of the div. But they are disappearing correctly when they hit the left or top. Strange. However, if you recall that the origin of an HTML element is the upper left hand corner of its rectangle, then this bug makes more sense. In order to resolve, we would have to overwrite some of MoveStar's calculations around resetting the image location to also account for the size of the img.

And then there's this horrible border thing. The border is definitely the result of setting the CSS properties for the Star and MoveStar classes directly in the JavaScript rather than in a separate CSS class that we can toggle on or off. If we move the styling of the Star and MoveStar (specifically the border-style, border-color, and border-radius) to a separate CSS file, then we wouldn't have these white borders on our TieFighters. For now, let's just pretend these boxes are from the Millenium Falcon's weapons targetting system.

The Force Awakens

At this point, we've now demonstrated how to use JavaScript subclassing to create the foundations of a Stars Wars or Star Trek type game. A big Chewbacca-style hug to Tate Thurston who was my co-conspirator on this project.

Rather than continue to walkthrough the code (because this post is long enough already), I'll instead list some considerations, challenges, and steps we faced in the transition from this visualization to our game:

• Millenium Falcon cockpit
• Use GIMP to create transparent png and overlay Han and Chewie cockpit image on top with CSS "z-index" property.
• Use animate.css to "shake" the cockpit image after a hit from a TieFighter (when the TieFighter reaches a z-position of zero), and/or flash the screen semi-transparent red with CSS.

Tiefighters

• Use CSS to flip the image based on initial x-position to make their flight pattern look more realistic.
• Turn the mouse cursor into a crosshair with CSS
• Use setInterval to consistently generate new Tiefighters
• Set an onClick listener to track succesful "hits", and then turn the Tiefighter img source into an explosion-type gif, and then remove the destroyed TieFighter. (Sidenote - this is more challenging than simply removing the DOM node, since the TieFighter object will continue to exist in memory even though the DOM node was removed. Frustratingly, we kept getting invisible "hits" from already-destroyed TieFighters, and we finally resolved by using a boolean flag to track whether a Tiefighter had been destroyed or not before iterating to the next call to the original Star "step" function).

Sounds

• Use one of the many free sound effects websites to create laser, explosion, injury, and other fun videogame sounds, and then trigger them via event listeners
• Find some background music and set on autoplay with an HTML audio tag

Hyperspace

If you've already tried the game demo, you may have noticed that the Millenium Falcon's hyperdrive is not operational. That's an obvious next step for future development!

I spoke in Berlin, and took a really bad still image, apparently.

Knerds is a simple and fun iOS game that helps you learn the names of your co-workers through brute force, cuddly mascots, and spaced repetition.

Diving into Objective-C

Like many tech companies, Knewton organizes a hack day every few months. Beyond the inevitable pizza gorging, the fun and hopefully useful projects, and the geeky camaraderie, hack day has always been important to me — discovering this 2011 Knewton hack day video more or less kept me alive as a functioning human organism during the darkest days of my pre-Knewton career.

I had just finished the Big Nerd Ranch Objective-C book when our April 2015 hack day rolled around, and I decided to try my hand at crafting a dead simple iPhone app.

My idea: Tinder for work. Well, not exactly.

I wanted to build a flashcard app that helped you learn the names of your co-workers with pictures and some randomized guessing action. Fun, potentially useful (especially for those of us in the London office), and quite possibly something I could pull off in a few days.

Pretty soon, after an hour or so on Thursday night, Knerd by Knewton lived:

It wasn’t a good app — the picture didn’t change, there were no swipes or gestures, and the “quiz” was pretty lame — but the basic elements were there. I had a foundation. I was able to test my app on my actual phone — awesome. Even if I wasn’t able to proceed much further, I knew that I would be able to submit at least a basic working iOS quiz app as my “hack.” Back to the drawing board

We use a Trello board to organize potential hacks into four categories: Performance, Product, Business, and Culture. The Trello board is also used to recruit fellow hackers to your pet project. Earlier that week, I dropped a vague card into the Product category called “Knerd by Knewton” with an even more mysterious description.

To my shock Dan McGorry, our lead UX designer replied:

And that’s when things got real.

UX

Dan and I chatted throughout Friday on Slack (him in New York, me in London) and we debated several challenges with the concept:

• What does a left swipe mean? A right swipe?
• Should the quizzing occur when the picture is still on the screen (and thereby losing the Tinder flicking concept, but reinforcing the memorization experience)?
• How should we handle incorrect answers?

Designing solid UX is more challenging than I thought. My initial concept for a Tinder-like flow was much more complicated than I had initially anticipated. Dan put together an early set of wireframes in blazing speed, and we continued to discuss how to improve the experience as he iterated on the wires.

Here’s a look at one of the early sets:

That Friday was not exactly a day off from work for me, and I told Dan as much — that we should continue to develop the wires and concept up to the end of the day, and that I would devote my weekend to implementing our app, but with no guarantee of my success — just the old hack day try.

Enter the knerds

Dan and I had never worked together on anything before, but we were really syncing up well on Friday. I don’t want to say that our collaboration was effortless, because we both put in quite a bit of work, but it was easy. And fun. That’s one of the real beauties of hack day, discovering similar interests and collaborating with folks outside of your day-to-day job.

Importantly, Dan and I agreed that the app was lacking something critical — a cuddly mascot. He showed me a Pinterest board he’d been collecting with mascot ideas, and I also mentioned my love for those armless french-fry guys in the McDonald’s world.

Pretty soon, Dan came back with the knerds:

We chatted through some minor tweaks, like the placement of bows, the eye colors, and the introduction of a fourth triangle-y knerd, but these little geometric critters were pretty much perfect from the start. And my love for the knerds began.

By the end of Friday, our wires looked like a real app:

Now it was time for me to code up or shut up.

Weekend of code

I spent the next 48 hours doing nothing but coding, reading, power-napping, and drinking coffee, with a few sanity breaks for running and grabbing pizza — and it was amazing. I made continual progress throughout the weekend, which was both exhilarating and nerve-wracking because I needed to get it all working by the Monday deadline.

Some challenges encountered along the way:

Moving between view controllers: The standard UITabBarController or the UINavigationController flows did not fit into our card-based app design, so I spent time exploring different flows for our game. Eventually, I used modal view controllers to toggle between the three screens (Start, Game, and Finish), with a UIAlertController for the quiz questions. Here’s how I present the Game Screen when a user clicks the “Let’s Go” button:

- (IBAction)playGame:(id)sender { 
 KDGameViewController *gvc = [[KDGameViewController alloc] init]; 
 [self presentViewController:gvc animated:YES completion:nil]; 
}

Once you’re in the KDGameViewController, the “finish/victory” screen is shown in a similar fashion. Then, when the user taps “Play Again”, the “finish” screen is dismissed, and the user returns to the Game screen (where everything is reset).

- (IBAction)playAgain:(id)sender { 
  [self dismissViewControllerAnimated:YES completion:nil]; 
}

Resetting the game: After the Finish screen, I wanted to draw the users back into the app to try the game again (hello, viral hook!), but the analytics counters were not properly resetting (my basic game analytics / metrics included number of hints, wrong choices, cards remaining, etc). I resolved this by utilizing the viewWillAppear() lifecycle method to reset all my counters. Randomizing the answer choices: This was a fun challenge — displaying randomized answer choices in the “Alert” pop-up. Here’s my naive but workable solution:

// Select the "correct" and "distractor" choices 
NSMutableArray *names = [[NSMutableArray alloc] init]; 
// name1 points to the "correct" name 
[names addObject:name1]; 
// name2, name3, and name4 point to the distractor names 
[names addObject:name2]; 
[names addObject:name3]; 
[names addObject:name4]; 
// Randomly pick one of the "names" in names, 
// then create a new NSString to hold that name, 
// and finally remove it from the names array 
int random1 = arc4random() % [names count]; 
NSString *nameTitle1 = names[random1]; 
[names removeObjectAtIndex:random1]; 
int random2 = arc4random() % [names count]; 
NSString *nameTitle2 = names[random2]; 
[names removeObjectAtIndex:random2]; 
int random3 = arc4random() % [names count]; 
NSString *nameTitle3 = names[random3]; 
[names removeObjectAtIndex:random3]; 
int random4 = arc4random() % [names count]; 
NSString *nameTitle4 = names[random4]; 
[names removeObjectAtIndex:random4];

At this point, I’ve got 4 NSString pointers called nameTitle1, nameTitle2, etc. that will randomly hold the “correct” answer choice every time — sometimes the correct choice will be nameTitle2, other times it will be nameTitle4, etc. All I then have to do is create UIAlertActions with titles that reference nameTitle1, nameTitle2, etc.

Feature creep: I kept coming up with ideas for fun new features and mechanics before I even had the base functionality working (e.g. adding score “analytics” to the finish screen). This is a real problem. At the same time, it’s great to feel that continuous inspiration, and I would never want to entirely constrain that creativity.

By 1:45 AM on Monday morning, Knerds was a working iOS game! Here’s a screencast that I recorded to submit the hack, just ahead of the Monday morning deadline (note that I hadn’t yet fixed the “repeated name” bug for the distractor answers, which I resolved later that morning, and also note that I say the word “sweet” a lot in this video):

I shouldn’t have recorded this at 1:42 AM on Sunday.

What’s next for Knerds

TestFlight: After some clunking around with iTunesConnect, I’ve now got a dozen or so actual knerds testing out the app via TestFlight. The swiping UX needs to be improved. It may just make more sense to go with a “double-tap to guess” concept. Also, I need to fix the positioning of the views for different phone sizes. I had quickly used the “Reset to Suggested Constraints” option in Interface Builder for each view object ahead of releasing on TestFlight, but I’ve heard back and it didn’t work out as well as I had hoped.

Release as an internal company app: This requires an enterprise account, and a Dun & Bradstreet number. My hope is that we’ll be able to release Knerds for all current and future Knewton employees as a helpful HR tool.

Automatic content refresh: I scraped the pictures, names, and titles from our company page on Employee-List. It doesn’t look like they have an API, so I’m exploring some options for automatic content updates as more folks join Knewton.

Game improvements: One obvious improvement could be better distractors — filtering by team or location.

I had a great time building this app with Dan, and I can’t wait to continue to develop Knerds!

One more thing

I flew to NYC later that week, and I couldn’t stop thinking about the knerds, and I decided to come up with a backstory for them. I quickly sketched up some storyboards on some scraps of paper during my flight, and emailed them to Dan to see what he could put together.

Here it is — the backstory of the knerds:

But Dan and I kept going. We named them all. And then I couldn’t stop myself — the knerds needed personalities:

Olive

Olive is the young Captain of the knerds. This mission was her first ever. Olive is brave and bold, and loves to explore, but she’s still young and inexperienced. She’s determined to make the most of their mission!

Mr. Figg

Mr. Figg is Head of Security. He’s responsible for protecting the Knerds from trouble, but somehow Isaak always seems to find a way to make it anyway. Mr. Figg has a tough exterior, but also a heart of gold.

K.A.M.

K.A.M. is the Communications Officer. She’s part robot (we think). Her suction cup legs can rotate around her body, so she can easily climb and stick to any surface. She also can upgrade her legs to new components, like helicopter drone blades so she can fly, but she lost her repair/upgrade kit during the crash.

Isaak

Isaak is Olive’s younger brother. He snuck onto the ship before takeoff because he didn’t want to miss out on the adventure! Isaak is always getting into trouble, which is easy because he’s so tiny, but his misadventures often lead the knerds to important new discoveries.

The continuing adventure

I think that the story of the knerds could be the basis for a great game for kids to learn maths (see, I’m living in Europe now) and vocab content on their phones, following along with the mission of the knerds as they relearn their skills and try to salvage their mission in time.

As validation, I showed the knerds to my eight-year-old cousin Avery over the weekend, and she came up with even more superpowers for them. Apparently, Isaak can turn into a tiny paper football. And Mr. Figg can grow his hair really really long. And Olive is actually a popsicle and she can melt and surf along her melted self and then regenerate.

Avery convinced me that helping the knerds solve puzzles would be a fun game for kids. She even started to memorize the names of the Knewton employees after repeatedly playing the app. Oh, and Avery invented bad guys for the knerds! I asked her to draw them:

Binky

Chill Factor

Bill

It looks like the knerds will need all the help they can get — will you help them?

Last week, streaming live from London, TERMINAL MAN performed its first show ever, appearing live in New York City at the Knewton Talent Show. This may have been the first musical performance for a Double Robotics robot ever, but I certainly hope it's not the last.

What follows may shock you:

Here's how I brought TERMINAL MAN to life.

The music

Garageband

I've spent a lot of time at airports over the last year and a half. When you have that much free time, you invent some hobbies. One of mine is creating loop-driven songs on Garageband.

This screenshot is one of my more recent tracks Over Water, Under You. I assembled July Nineteen in a similar fashion, playing with the pre-set available loops to come up with weird and interesting combinations. I decided to use 'July Nineteen' for the talent show performance because it's loud and short and catchy.

The video

The skull

• Wikimedia Commons
• GIMP

When I was growing up, my dad called me "Skull" as a nickname. To this day, if someone yells "Skull!", I will come running. Anyway, I wanted a giant skull on giant screen, so I grabbed a decent skull image from Wikimedia Commons and then doctored it up in GIMP to remove clutter and the white background, saving as a .png.

The background

• Processing
• YouTube
• CloudApp

I've always loved the weird video backdrops behind live performances of My Morning Jacket or Wilco or Sufjan or Animal Collective. Who creates these trippy backgrounds, and how much input comes from the band?

Hey, so, when we're playing "I'm The Man Who Loves You" I want the background to be like crying rainbow waves. And triangles, lots of triangles.

Well, TERMINAL MAN definitely needed its own trippy background for its debut live show, and Processing made it happen.

I created a Processing sketch that used keyboard strokes to trigger different backdrops behind the giant skull. Some of the backgrounds were native Processing functions (like creating random flashing colors or drawing lots of random lines), and others were looped video clips that I grabbed from YouTube with CloudApp. The central theme: the joy of Windows 95.

Here's the code for the sketch:

// ----------------------
// T E R M I N A L  M A N
// ----------------------
//   @ w h a t r o c k s
// ----------------------

// import video library
import processing.video.*;

// movies
Movie pipeMovie;
Movie doomMovie;
Movie solitaireMovie;
Movie toastersMovie;
Movie mazeMovie;
Movie wc2Movie;
Movie tiefighterMovie;
Movie rollercoasterMovie;

// images
PImage skull;
PImage miniSkull;

// font
PFont f;

void setup() {
  size(800, 800);
  smooth();

  // load the pictures
  skull = loadImage("skull2.png");
  miniSkull = loadImage("skull2.png");
  // resize the big skull to dimensions of the canvas
  skull.resize(width, height);

  // black background
  background(0);

  // load the font
  f = loadFont("TwCenMT-BoldItalic-100.vlw");

  // load the movies
  mazeMovie = new Movie(this, "maze.mov");
  mazeMovie.loop();

  solitaireMovie = new Movie(this, "solitaire.mov");
  solitaireMovie.loop();

  doomMovie = new Movie(this, "doom.mov");
  doomMovie.loop();

  pipeMovie = new Movie(this, "pipe.mov");
  pipeMovie.loop();

  toastersMovie = new Movie(this, "toasters.mov");
  toastersMovie.loop();

  tiefighterMovie = new Movie(this, "tiefighter.mov");
  tiefighterMovie.loop();

  wc2Movie = new Movie(this, "wc2.mov");
  wc2Movie.loop();

  rollercoasterMovie = new Movie(this, "rollercoaster.mov");
  rollercoasterMovie.loop();

}

void draw() {

  // black background
  if (key == 'x') {
    background(0);
  }

  // play doom video
  if (key == '1') {
    image(doomMovie, 0, 0, width, height);
  }

  // play maze video
  if (key == '2') {
    image(mazeMovie, 0, 0, width, height);
  }

  // play toasters video
  if (key == '3') {
    image(toastersMovie, 0, 0, width, height);
  }

  // play pipes video
  if (key == '4') {
    image(pipeMovie, 0, 0, width, height);
  }

  // play solitaire video
  if (key == '5') {
    image(solitaireMovie, 0, 0, width, height);
  }

  // play tiefighter video
  if (key == '6') {
    image(tiefighterMovie, 0, 0, width, height);
  }

  // play rollercoaster video
  if (key == '7') {
    image(rollercoasterMovie, 0, 0, width, height);
  }

  // play wc2 video
  if (key == '8') {
    image(wc2Movie, 0, 0, width, height);
  }

  // print mini skulls everywhere!
  if (key == 's') {
    miniSkull.resize(width/10, height/10);
    image(miniSkull, random(-width, width), random(-height, height));
  }

  // flashy background colors!
  if (key == 'b') {
    background(int(random(255)), int(random(255)), int(random(255)), int(random(255)));
  }

  // draw blue lines from the center
  if (key == 'l') {
    stroke(0, random(200, 255), random(200, 255), 150);
    line(width/2, height/2, width/2 + random(-width, width), height/2 + random(-height, height));
  }

  // Load big skull picture on top of all these other backgrounds
  image(skull, 0, 0);

  // show 'terminal man' text, even on top of the big skull
  if (key == 't') {
    textFont(f);
    textAlign(CENTER);
    fill(random(155, 255), 0, random(150, 255));
    text("TERMINAL MAN", width/2, height/2);
    fill(0, random(0, 255), random(150, 255));
    text("TERMINAL MAN", width/2+10, height/2+10);
  }
}

// Called everytime a new frame is available
void movieEvent(Movie m) {
  m.read();
}

The sketch creates an app that lets you use the keyboard to jump between the different backdrops (for example, I can press 'b' for crazy random flashing colors, and '5' for a looped Solitaire video). Note that the media files need to appear in the 'data' folder of your sketch for this to work correctly.

Pulling it all together

• QuickTime
• iMovie
• Vimeo

Even though the sketch works great as a live companion to the song, I was really concerned with potential lag and getting out of sync with the music, so I decided to record it all as a video. CloudApp stopped cooperating, conveniently right after I registered for the one-month premium package, so I Googled and discovered that I could also use QuickTime to record screencasts. For free.

I put on some DJ-like headphones, press record on the Quicktime screencast, pressed play on the track in iTunes, and just pretended that I was Chris Kuroda, the de facto fifth member of Phish who runs all their stage effects and lights, as I recorded the background changes along with the song.

Then all I had to do was drop the mp3 and the video files into iMovie, line them up, and post to Vimeo!

The live performance

• Double Robotics robot (we call it the Krobot at Knewton)
• Terminal
• Stage crew (Fernando, Sam, and others!)

I had elaborate plans to introduce myself in a scary and funny robot voice using the Terminal 'say' command. This totally failed, however, as you can see in the video. No one could hear my hilarious introduction. No one heard me explain that I wrote the song and created the background video.

But none of that mattered once I started to dance.