Apr 17, 2015

Test drive: real-time prediction in Java with Amazon Machine Learning

Following up on my two previous Amazon ML articles (batch prediction with Redshift and real-time prediction with the AWS CLI), here's a quickie on implementing real-time prediction with the AWS SDK for Java.

At the time of writing, I'm using SDK version 1.9.31:
Here's the Maven dependency:

The code (source on Github) is pretty self-explanatory and totally similar to the CLI example in the previous article.
  • Get the list of models
  • Pick one
  • Build a prediction request : data record + endpoint
  • Fire the request
  • Get a result!

Here's the output:

Nice and simple. What about performance? My previous article measured a CLI call (aka 'aws machinelearning predict') within us-east-1 at around 300ms, well above the 100ms mentioned in the FAQ.

Believe or not, the Amazon product team got in touch (I'm not worthy, I'm not worthy... and thanks for reading!). They kindly pointed out that this measurement includes much more than the call to the web service and of course, they're right.

Fortunately, the Java program above is finer-grained and allows us to measure only the actual API call. I packaged it, deployed it to a t2.micro EC2 instance running in us-east-1 (same configuration as before) and...

Average time is around 80ms, which is indeed under the 100ms limit mentioned in the FAQ. There you go :)

I'm definitely looking forward to using this in production. Till next time, keep rockin'.

Apr 16, 2015

Test drive: real-time prediction with Amazon Machine Learning

As explained in my previous article, Amazon ML supports both batch prediction and real-time prediction.

I burned some rubber (and some cash!) on batch prediction, but there's still some gas in the tank and tires ain't totally dead yet, so let's do a few more laps and see how the real-time thing works :)

Assuming you've already built and evaluated a model, one single operation is required to perform real-time prediction: create a real-time endpoint, i.e. a web service URL to send your requests to.

Of course, we could to this in the AWS console, but why not use the CLI instead?  A word of warning: at the time of writing, the CLI package available from the AWS website suffers from a nasty bug on prediction calls (see https://forums.aws.amazon.com/thread.jspa?messageID=615018). You'll need to download and install the latest version from Github:

While we're at it, let's take a quick look at the CLI for Amazon ML. Here's our model, built from 100,000 lines with the format described in my previous article:

Let's also look at the evaluation for this model. The log file URL gives away the fact Amazon ML jobs are based on EMR... in case you had any doubts ;)
  All right, everything looks good. Time to create a prediction endpoint for our model:

Pretty simple. We can now see that our model is accessible at https://realtime.machinelearning.us-east-1.amazonaws.com

And now, the moment of truth: let's hit the endpoint with a record.

Boom. We got a predicted value. This actually worked on the first call, I guess caffeine was stronger this morning ;)

One last question for today: how fast is this baby? I tested two different setups :
  • Return trip between my office in Paris and the endpoint in us-east-1 : about 700 ms
  • Return trip between an EC2 instance in us-east-1 and the endpoint in us-east-1 : about 300 ms

(and yes, I did run the tests multiple times. These numbers are average values).

Slower than expected. The Amazon ML FAQ says:

Q: How fast can the Amazon Machine Learning real-time API generate predictions?
Most real-time prediction requests return a response within 100 MS, making them fast enough for interactive web, mobile, or desktop applications. The exact time it takes for the real-time API to generate a prediction varies depending on the size of the input data record, and the complexity of the data processing “recipe” associated with the ML model that is generating the predictions

300 ms feels slow, especially since my model doesn't strike me as super-complicated. Maybe I'm jaded ;)

Ok, enough bitching :) This product has only been out a week or so and it's already fuckin' AWESOME. It's hard to believe Amazon made ML this simple. If I can get this to work, anyone can.

Given the right level of price, performance and scale (all will come quickly), I see this product  crushing the competition... and not only other ML SaaS providers. Hardware & software vendors should start sweating even more than they already do.

C'mon, give this thing a try and tell me you're STILL eager to build Hadoop clusters and write Map-Reduce jobs. Seriously?

Till next time, keep rockin'.