New York Tech Journal
Tech news from the Big Apple

State of Deep Learning – Neuron inspired Artificial Intelligence

Posted on October 22nd, 2014

10/22/2014 @RRE Ventures, 54 W 21st, NY, @BotsAndBrains

Rob Fergus @Facebook and @NYU talked about an application of #NeuralNets to #ClassifyImages. He talked about deep learning which is built on a hierarchy of feature extractors. In the past hierarchies were used, but the lower level feature detectors were coded and only the top level was trained. Current methods now train many levels simultaneously.


In Convolutional neural nets, up to 20 levels of processing are done.

At each level: filters are applied at the pixel level -> nonlinear functions -> spatial pooling

  1. Convolution: Use 100’s of filter applied at each point in the picture. Each filter produces a feature map
  2. Non-linear function: apply a nonlinear function to each pixel in the feature map (one example is a rectified linear function : output = max(0, input) )
  3. Pooling: In the pooling stage, one specifies windows of a fixed w & h. (e.g. 3×3) and outputs the maximum value over elements within that window.

Next go to the next higher level and repeat these steps

Rob next talked about the recent progress in image recognition. Until 2012, not enough images were available to train the recognition on anything more complicated than a simple object in an uncluttered background.

In 2012, imageNet contributed a large image library and recognition improved by applied GPU technology with lots of parameters. Since recognition of more complicated images has improved further and in some cases approaches the accuracy of humans.

Rob concluded by talking about the limitation on the current bottom-up process which does not make use of top-down reasoning.: humans appear to use both bottom-up and top-down methods, but experiments can be conducted to suppress in top-down processing. This is done by flashing images very briefly (100ms) so there is not enough time for a feedback loop. Experiments appear to show that when top-down processing is suppressed humans and ConvNet identification errors are comparable. has an online demo of Deep Learning. The following image shows the limits of the method as the ‘hidden Dalmatian’ is not recognized by the program.

Clarifai analysis of hidden dalmation

posted in:  Bots and Brains NYC, data analysis    / leave comments:   2 comments