This past summer I set a goal for myself of finishing the Coursera Machine Leaning course taught by Andrew Ng. The class has been running for several years now and it’s been something I’ve thought about doing since the first one but it wasn’t until this summer that I made the commitment to myself to sit down and do it. This was my first (and so far only) experience with Coursera as a platform and unfortunately, it really left a lot to be desired.

From an infrastructure perspective, the website itself really left a lot to be desired, especially from a company that has received $85 million in funding. Page load speeds felt really sluggish and even when they did load, I was presented with a nice loading spinner signifying that I had only downloaded a skeleton webapp that then needed to bootstrap and download the rest of the content. I can understand this behavior if it only happens the first time I visit the site but this is not the case with Coursera. Internal links also result in this spinner which is particular frustrating because the content won’t fully load if the tab is in the background. For those of you that like to open new tabs, wait for the page to load, then view the content, you’ll feel my pain.

Another surprisingly large pain point for me was accessing the course wiki. This was often frustrating because the wiki’s login system is not intimately tied with that of the main webpage. Clicking the “course wiki” link in the main class navbar opened a new page which then asked me to login again to see the content. Even more annoying, the auth cookie is only set for the session meaning I constantly had to re-login to the wiki site throughout the duration of the course. For those of you thinking “what a minor and unimportant thing to care about”, if your friend’s weekend hackathon project can save my session indefinitely, why can’t Coursera?

As for the actual content of the course, I was pretty satisfied with the material covered. Topics included various techniques in both supervised and unsupervised learning and I most certainly walked away with a better understanding of what exactly machine learning is and where it’s useful. From what the course description tells me, there are typically different levels of involvement people will have ranging from only watching the lectures to completing all the assignments and doing the review questions. I fell into the latter camp and every week, I’d sit down in front of my computer, fire up Octave, and begin trudging through the assignments.

When I first started the course, I was really enthusiastic and would watch all the lecture videos before attempting any of the review questions. As the course wore on, however, I found myself slowly slipping back into my old college habits. Often times, life would get in the way of being able to dedicate a consistent amount of time each week to my studies and throughout the 10 week course, I slowly began skipping through the lectures, before eventually ignoring them altogether (which I suppose is the equivalent of skipping class). Instead, I would skim through the wiki pages to get an overview of the material and pound on the review questions until I got them all correct (thus learning the material through trial and error).

Of all the various elements of the course, I found working through the programming exercises to be the most informative and rewarding. As a professional programmer and someone who actually paid attention to Linear Algebra in high school, I spent minimal time wrestling with the Octave syntax and often times found myself vectorizing equations without really thinking about it (parts of the course were dedicated to re-writing iterative solutions in vector form for performance reasons). Although It was nice to dust off those matrix manipulation skills after several years, I couldn’t help but feel like many of the assignments suffered from the “draw the owl” effect. No matter how many times Professor Ng told me I “should now be an expert in machine learning” after “building” a functional movie recommendation system, I couldn’t help but feel that I had little idea how I had arrived there.

Courera has always been an interesting website to me since I truly believe in the vision of free knowledge for everyone. Even now, I get excited browsing through the course listings and dreaming of learning everything from bio statists to macro economics. Completing the ML class has made me realize that Coursera really is about bringing the large lecture University experience to the masses via the Internet. It also reminded me why I chose to graduation college early myself: because that experience just wasn’t compatible with how I learn best.

Recently, I’ve been developing a custom static site generator using gulp. Among other things, one of the requirements of the site is to automatically process multiple markdown files into webpages for inclusion on the site. As an author of long markdown documents, it’s often convenient to split these documents into multiple files and give them names with numbers to signify what order they should be assembled then. This presents a problem, however, because computer programs tend to sort strings in ASCIIbetical order meaning they are sorted by comparing character ASCII values to each other.

1
2
3
4
5
6
7
8
9
10
11

$ ls -1
part-1.md
part-10.md
part-2.md
part-3.md
part-4.md
part-5.md
part-6.md
part-7.md
part-8.md
part-9.md

When we try and feed these files through our gulp pipeline, part-10.md will be inserted between part-1.md and part-2.md. The way to get around this problem is by using something called a “natural sort” which attempts to sort names in way a human would do so. There are many different implementations of this sort that will try to recognize all sorts of things from dates and times to IP addresses. The sort is not always perfect but for my purposes of sorting alphanumeric strings in a logical way, it works just fine.

In order to incorporate this sort into my gulp pipeline, I created a quick gulp plugin that when applied to a stream, will re-emit the files in a naturally sorted way. It uses the natural-compare-lite library but can be easily modified to use any other sorting algorithm.

Head on over to npm to check it out or view the source code over at github.

I recently started on an Ionic/Cordova app that uses Gulp as the main build system. While I’ve used Grunt several times in the past I have found Gulp to be much more of an intuitive approach and definitely easier to work with.

In a Cordova project, you develop the application in the $ROOT_DIR/www/ directory. When you build the project for ios, Cordova copies those files from that directory to $ROOT_DIR/platforms/ios/www directory. In order to streamline the development process, I wanted to make use of
livereload via the gulp-livereload and cordova-gapreload plugins. The basic setup is to use Gulp to watch for changes in the www/ directory and when a change is detected, copy the file to the appropriate location in the platforms/ios/www directory before triggering livereload. The main reason for copying files is that the gapreload plugin looks for assets served relative to the platforms/ directory.

As you’d expect with any project, asset files can be numerous and the build system itself should not impose a restriction on nested folders. These requirements make manually listing files and structures an onerous task that should be avoided. Fortunately, file globbing) exists precisely to simplify this task. When specifying files to watch, you can use a glob pattern to match multiple files in many directories with a single expression. For example, to watch all files in the www/ directory, you could specify a route glob as such: www/**

Important: Pretty much every JS library I’ve found that does any sort of route matching with globs uses the minimatch library. Currently this library has what I consider to be unintuitive, undocumented, and even incorrect behavior. The library does not do path normalization before matching which leads to the library not recognizing the equality of paths prefixed with the relative “./“ compared to those that are not. Thus a glob specified as ./www/**/*.js would NOT match the path www/app/test.js . In the Gulp ecosystem at least, it seems that path names are piped between processes without the “./“ so I would recommend that any relative paths or globs you manually specify not have the “./“ prefix.

Once you have Gulp setup to properly watch your asset files and copy them to the appropriate directory, the next step is to configure a basic http server to serve those assets, then trigger a livereload. I fulfilled the first requirement using the simple ecstatic module configured to serve all files from the platforms/ios/www/ directory on port 8000. Make sure your assets are served with a very short cache time . This can be done with ecstatic by specifying cache: 0 in the config object. Once Gulp copies a changed file to the appropriate directory, pipe the new file to livereload to trigger the change.

The final step is to install the the Cordova plugin. From your root directory type: cordova plugin add pro.fing.cordova.gapreload --variable SERVER\_HOST="<host>" where is the is the computer the files are hosted from. If you’re developing with the emulator, this would probably be localhost however you can also specify any IP you want meaning you can use this setup when developing with an actual device

Conclusion That’s pretty much it! Now when you save a file, Gulp should

automatically detect the file has changed, copy it to the platforms directory, then trigger a livereload action. You app running in the emulator should instantly receive the livereload notification, load the changed file from the server, then refresh the webview so the changes take effect.

Caveats This setup has been working pretty well for me however there are

some caveats that are important to mention. First of all, when using the gapreload plugin XMLHttpRequests will be subject to cross origin restrictions meaning you won’t be able to make normal requests to servers that don’t have CORS enabled, even if you whitelist the domain in your config.xml.

Below are the relevant portions of my Gulpfile.js:

Introduction

The HTML audio element is pretty powerful, unfortunately, mobile browsers tend to restrict it by not letting you play more than one audio stream at once. Furthermore, these browsers will also restrict buffering multiple audio tracks making seamless audio switching difficult at best. While these restrictions are fine for most applications, some apps require many sounds to be played in quick succession. Games with sound effects are the biggest victim of this policy and the most commonly used strategy to circumvent it is to use audio sprites. Below, I will discuss an alternate method that makes use of XMLHttpRequests and the experimental Web Audio API to buffer audio files in parallel then seamless switch between them.

Solution

The first thing we need to do is setup our Web Audio API context. Because the spec has yet to be finalized, it may be necessary to use a vendor prefix:

1
2
3
4
5

if ('webkitAudioContext' in window) {
  var myAudioContext = new webkitAudioContext();
} else {
  var myAudioContext = new AudioContext();
}

Here, we simple check to see if the prefixed version exists, if not we use the non prefixed version.

The XMLHttpRequest object has received some major updates in the last few years. In addition to being able to send and receive text, it can now do the same with arraybuffers, blobs, and documents. Using the wonderful Async.js library, we simultaneously download multiple audio files as arraybuffers and save them for playback later:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

var songUrls = [ /* list of song URLs to buffer */ ];
var songData = [];
async.map(songUrls, function (url, cb) {
  request = new XMLHttpRequest();
  request.open('GET', url, true);
  request.responseType = 'arraybuffer';
  request.addEventListener('load', function (event) {
    try {
      var request = event.target;
      var source = myAudioContext.createBufferSource();
      source.buffer = myAudioContext.createBuffer(request.response, false);
      return cb(null, source);
    } catch (err) {
      return cb(err);
    }
  }, false);
  request.send();
}, function (err, result) {
  if (err) {
    $('#message').text("Error Buffering: " + err);
    return;
  }
  songData = result;
});

You’ll notice that once the array buffer is finished downloading we immediately create a new AudioBufferSourceNode and populate it with the downloaded data using the createBuffer method (bonus: the preferred way to do this step is to use the decodeAudioData() method because it’s asynchronous and has better error handling).

At this point, we now have an array of audio buffers so let’s look at how to play them back. The Web Audio API is based on the concept of nodes which allow you to channel data through various transformation functions before finally outputting it. For the sake of this tutorial, all we care about is playback so all we need to do is connect one of the buffers to the output device. We do that using the connect() method:

1

songData[0].connect(myAudioContext.destination);

Although we can start playing the audio now, it is not ideal to do so because audio buffers cannot be restarted. To get around this, we first clone the audio buffer we want to playback:

1

source = myAudioContext.createBufferSource(); source.buffer = songData[0].buffer; source.connect(myAudioContext.destination); source.start(0);

Now, if we want to restart the track or switch to a new track, we can do so by calling source.stop(0) then clone the next audio buffer and start playing it.

A working implementation of this method can be found at http://emgeee.com/projects/web_audio_test

Limitations

While the method described overcome some of the limitations imposed by mobile browsers, it is still constrained by the need for some sort of user interaction before the audio can start playing. On the bright side, we can at least buffer the audio before hand so that when the user does hit play, the audio will start immediately. Normal HTML5

Unlike the

Conclusion

This post describes a method for buffering audio in parallel and playing it back using the experimental Web Audio API. I expect this method will become more robust in the future as the API is further standardized across browsers. I have tested this solution on both Chrome 35 for Android and Mobile Safari running on IOS7 with great results. Unfortunately, the stock Android browser does not support Web Audio, which is particularly unfortunate because that is what is used to run hybrid Cordova apps. For those looking to target IOS exclusively, however, I see this as a viable solution.

Useful links

• https://developer.apple.com/library/safari/documentation/audiovideo/conceptual/using_html5_audio_video/PlayingandSynthesizingSounds/PlayingandSynthesizingSounds.html
• https://developer.mozilla.org/en-US/docs/Web/API

Full Sample Code

Introduction

The other day I was attempting to install powerline to make my Vim setup look prettier when I ran into some trouble with how my system was configured. For those who don’t know, powerline is a neat status bar written in Python that provides various visual cues like what mode you are currently in. The instructions for Mac told me install the package via pip, a Python package management system. This sounded easy enough however I quickly ran into some nuanced issues that had to be resolved.

Default Python on Mac

By default, OSX comes preinstalled with several versions of Python. Theses version are used as dependencies for various software and, in general, should not really be messed with. The executables are made available via the /usr/bin directory and should link back to /System/Library/Frameworks/Python.framework/Versions/2.7/bin/ . This is all well and good, however, there are a few problems: firstly, the version of Python in this directory is not necessarily up to day. For me, /usr/bin/python —version yields version 2.7.5 while the most recent version is 2.7.6. More importantly, installing Python packages via the included version of pip require root access (sudo pip install ) because packages are saved to /Library/Python/2.7/site-packages. Automatically running install scripts that are downloaded form the internet with root privileges is generally considered a possibly security risk and is something I strive to avoid at all cost.

Enter Homebrew

To rectify both these issues, I used the wonderful Homebrew (brew) package management system. By design, brew promises to never write files outside the directory specified by brew —prefix which defaults to /usr/local/. Package information is saved to /usr/local/Cellar and executables are linked to /usr/local/bin . The beauty of this is that these directories don’t require root privileges to write to meaning if you ever need to sudo brew anything, you’re probably doing something wrong.

To install an updated version of python, simple enter brew install python. Because brew will detect you already have a version of python installed on your system, it may be necessary to run brew link python, which will symlink the appropriate executables to /usr/local/bin . Now, assuming you have /usr/local/bin in your $PATH (which you should), invoking which python should show you that you are now running the brewed version of python and python --version should now yield 2.7.6.

Rejoice! You should now have an updated version of python installed and managed by brew. which pip should also show you are running the version that accompanied the brewed python.

Installing powerline

As mentioned earlier, powerline can be installed using pip. The exact command given by the documentation is pip install --user git+git://github.com/Lokaltog/powerline.

Unfortunately, this presents an issue for brewed python. According to the brew-python documentation at https://github.com/Homebrew/homebrew/wiki/Homebrew-and-Python. The normal pip install --user is disabled for brewed Python. This is because of a bug in distutils, because Homebrew writes a distutils.cfg which sets the package prefix. Fortunately, the fix is quite simple. If you’ve installed python via brew, you can safely remove the —user flag, which instructs pip to install the package into the ~/Library/Python/<x>.<y>/lib/python/site-packages/ directory. Instead, pip will install the package into /usr/local/lib/python2.7/site-packages, which shouldn’t require root privileges.

Configuring Vim

The final step is to configure Vim (or whatever else you’d like to user powerline for). If, like me, you use vim or macvim installed through brew, you may have to do one last step. In order for powerline to work, Vim needs to be compiled with python support. Brew does this automatically at install time howeve, it relies on the first version of python found in the user’s $PATH when the program is installed. If you did not have brew installed python at the time you installed vim or macvim, you will need to reinstall these versions using something like brew reinstall macvim . Vim will now be linked against the correct version of python and have access to the installed modules.

You should now be able to add the following lines to your vimrc file and bask in the glory of a nifty status bar

1
2
3

python from powerline.vim import setup as powerline_setup
python powerline_setup()
python del powerline_setup

Those are some of the things I learned while wrestling with brew and python for several hours. I have been using brew for a year or so prior to this without necessarily understanding some of the finer points of the system and no doubt had a lot of cruft on my machine as a result. Hopefully something in this post can save someone out there some time and/or headaches.

For anyone how hasn’t heard of the Ergodox keyboard it represents the state of the art in keyboard technology, at least if you don’t care about having a mini OLED screen embedded in each key. The Ergodox has 3 qualities that make it a keyboard unique: it’s physically separated into two halves, it adds new keys to form two distinct thumb clusters (similar to the Kinesis), and its design is fully open source. All of these qualities combine to form a keyboard that is great for someone who regularly sits in front of a computer long enough to figure out how to actually build one — it’s ergonomically sound and every key is reprogrammable. I ordered one off of Massdrop; here’s how I built it:

The components arrived in a box unassembled and cushioned by bubble wrap. I opted for the cherry MX blue switches and the extended palm rest for this build. Although I would have preferred brown switches, supply issues made is so that those weren’t an option in the particular drop I participated in and I wasn’t in the mood to source switched myself. Besides, if I was going to assemble the keyboard anyways, I figured I could always unassembled it later (or just buy a new one).

Because the build was going to require a good amount of soldering, I opted to splurge a little bit on a decent soldering iron. I picked up a Weller WES51with adjustable temperature for around $80 on Amazon and a generic solder sucker for the occasional screwup. Because this build requires soldering surface mount diodes, I highly recommend using a quality rig and if you can get a fine solder tip, that’s even better.

The build began by soldering on a few resistors, headers, the headphone TRRS jacks, and the Teensey micro-controller that serves as the brains of the keyboard. The Ergodox PCB is designed such that both halves are actually copies of the same PCB so one arbitrary choice later, I was happily on my way to a semi-populated board. The kit contained two mini-B cables, one for connecting the keyboard to the computer, and another for connecting the Teensy micro-controller to the PCB. In order to accomplish the second task, the cable had to be cannibalized so the connector would fit into the casing. Doing so proved to be the most difficult (and dangerous) part of the entire build. I opted to slowly whittle away at the insulation with a sharp knife, however, I have heard that a Dremel is the most effective method.

!(http://1.bp.blogspot.com/-koxGLpIjPFk/U1787Ru2O6I/AAAAAAAAAV4/5OxWYNbfu28/s1600/IMG_20140428_171950.jpg)

The next part is probably the part most people have trouble with. The plans call for surface mount diodes to be painfully soldered next to each switch on the PCB. Needle tweezers are a must in this situation and I found it easiest to place a small dab of solder on one of the pads, then hold the diode over the pad while re-melting the solder and moving the diode into the proper position. Solder flux would have been very helpful in this situation however, I did not have any on hand so I opted to charge ahead without it. During this step, it’s absolutely important to ensure that each diode is facing the proper direction . Failing to do so will render the keyboard inoperable and you will have to go back and correct each incorrect diode. For this build, the tiny grey stipe on the diode should be facing the square pad.

The next step is to begin mounting each switch onto acrylic frame. Because the switches included in this kit are of the plate mounted kinda, as opposed to the PCB mounted types, this step must be done before soldering them on the board. This also means that if you want to change the case, you will have to desolder every switch in order to fully liberate the PCB. These plate mounted switches tend to be more durable than the alternatives, so that’s a win for stability, still, it makes me sad that once you continue past this step, you’re locked into using the original acrylic case.

I found it best to mount and solder each of the corner switches first before filling in the middle. This makes it easier to line up the switches with their respective holes. After this step is finished, be sure to inspect your work and make sure every lead has enough solder on it and no solder bridges exist that could short circuit your keyboard. At this point, you should be able to connect the two halves of the keyboard and plug it into your computer. You’ll have to flash the Teensy with the appropriate firmware but after doing that, you should have a fully functioning keyboard, all that’s left is to finish assembling the case and installing the keycaps.

The end result: my fully assembled Ergodox!

According to the official website, the Chatuchak market is one of the largest weekend markets in the world. It’s open every Saturday and Sunday from 6am-6pm and offers a wide variety of products for sale from terribly made trinkets to surprisingly high quality clothing. Thus far, I have ventured to this market twice and although I haven’t walked by every stall, I’ve managed to get a general idea for the kinds of things you can find there.

The easiest way to reach the market is by taking the BTS sky train all the way out to the Mo Chit stop. Exiting the train it becomes apparent that most everyone will also be heading to the market as it’s quite a popular destination on weekends. Once you reach street level, there is the normal smorgasbord of people peddling wares. The easiest thing to do is make an immediate right into a small park that lies adjacent to the market. You can walk through this park and gain easy access to the outer stalls.

The market itself is arranged in rows of covered stalls that extend the entire length of the place. In between each row are streets that are crowded with people and vendors selling all manner of items. Plenty of booths lie inside each row and because of the large nature of the market, it is easy to become disoriented and potentially lost.

We entered the market from the east side, which is where the nicest clothes appear to be sold. Most of the places in this section don’t appear willing to barter but the occasional blast of A/C from one of the stalls can be refreshing. If you’re looking for cheap knock-offs, it’s best to head deeper into the market.

One thing I noticed as I started looking at the various booths is that often times different people will be selling the exact same item. If you aren’t able to negotiate a price you want for a certain item, often you can just leave and try again at a place right down the row. This gives the illusion of variety in a huge market but the amount of stalls selling unique items is substantially lower than the total amount of stalls. Even something as seemingly remarkable as this crab sculpture could be found at multiple stalls in the market.

While the market sells a lot of cheap consumer items, there also exists a somewhat darker side to the market. Towards the west end, there is an area where booths sell all kinds of live animals from puppies to turtles. While cute and seemingly cuddling, the puppies and kittens for sale scream “puppy mill.” The place is definitely worth a walk through simply to see the somewhat unique animals on sale but be warned: it can be quite stuffy and could trigger allergies.

Overall the market is a great place to buy cheap clothes, trinkets, and other assorted crap and is definitely worth a visit. Contrary to what I’ve read at other places online, you probably don’t need more than a few hours to really experience the market because plenty of places are really just selling the same things. As with all touristy places, be wary of pick-pockets and know that bartering is essential to getting a good deal.

You can find more about the market at their website: http://www.chatuchak.org/

It’s been a little over a week since I first touched down in Bangkok and I’d like to share some thoughts about the experience from a first-time traveler’s perspective. To begin with, JFK is a terrible airport. I mean seriously, no free wifi? Also no Starbucks past the security line means I had to get my caffeine fix from something in a can. Also don’t fly China Eastern. The seats were far too small and the inflight entertainment was atrocious (read: non existent). It was also a weird experience seeing people milling around in the aisles mid flight chatting in Chinese across the width of the plane. I don’t feel like this behavior would have gone over too well on an American airline… oh well.

After an hour long delay in Shanghai and a flight that seemed to go on forever (you know the ones where you swear you’re almost there only to realize you’re 2 hours into a 4 hour flight), my friend and I touched down in Bangkok at around 4am in the morning. For whatever reason customs was the part that I was most worried about because I had absolutely no idea what to expect. Turns out expecting nothing would have been the right choice as a stamp and a smile later I was through to baggage claim.

Taxis in Thailand are a peculiar thing, or at least completely different than in America. Walking out of the airport we were hounded by private car drivers trying to get us to pay 700 baht (about $24 USD) for a car into the city. Fortunately, my somewhat more experienced comrade knew the actual price of things and we settled for a colorful cab that charged us a mere $10 USD to take us to our residence.

Some things to note about Bangkok: it’s a very modern city with excellent cell service at ridiculous prices and more malls than seems reasonable. I’ve been to more different malls in one week in Thailand than I have over the course of the rest of my life combined. In addition, I’m told the unemployment rate is around 0.3%, which makes sense considering how many people are on the street selling goods. If you don’t have a job simply set up shop on a street corner and start selling things.

Transportation

The city has two major metro systems, the BTS or “sky train” and the MRT which is underground. In addition, taxis are available for cheap as are scooter taxis, which are a truly dangerous adventure to use. To date, I’ve ridden one twice and have never gripped something as hard as I gripped the handles on the back of the seat. When riding one, it’s advised to keep your knees inside the width of the handle bars or risk loosing them when your driver decides to squeeze between a moving truck and a parked car. Of course you see native Thais riding side saddle talking on their cellphones while their drivers expertly dodge death but as someone who was raised to know what a helmet is, the 20 baht price tag isn’t enough to convince me to make it a regular thing. (Idea: someone should make a shirt that says “I survived a ride on a motor taxi in traffic.” Seriously, jumping out of an airplane isn’t as scary as swerving into oncoming traffic on the back of a Vespa)

Food

There’s really no which way about it: the food in Thailand is cheaper and better than that in America. Bubble Tea costs a little over $1 USD, chicken and rice costs around $2, and all you can eat Shabu runs slightly north of $11. If going out to eat all the time isn’t really you style you can also enjoy a variety of surprisingly delicious frozen food. Buy them at the grocery store for a buck or two or at a local 7-11, where they’ll even microwave it for you. Overall the food is quite delicious and available everywhere.

Closing Thoughts

Overall Thailand has been a wonderful experience thus far. I still have many more weeks to really experience it and plans to travel outside of Bangkok have already been concocted. Stay tuned to find out more about how you can use Node.js and Elastic Beanstalk to easily scale your webapps… errrr things in Thailand. I will leave you with a picture of the Bangkok skyline taken at sunset:

Greetings!

I have been meaning to start a blog for sometime now about my various adventures with technology. Through it, I hope to catalog my experiences solving difficult problems and building fascinating things. My hope is that these posts will help others learn from my mistakes and find solutions to their own problems.