New York Tech Journal
Tech news from the Big Apple

CodeDrivenNYC: #Web #Annotation, #NeuralNets #DeepLearning, #WebGL #Anatomy

Posted on December 17th, 2015


12/16/2015 @FirstMarkCap, 100 5th Ave, NY

Three speakers talked about challenging programming problems and how they solved them

20151216_181952[1] BioDigital20151216_185209[1] 20151216_190526[1]

Matt Brown @Genius talked about how they implemented their product which allows users to annotate text on web pages. The challenge is locating the text that was annotated on a web page and the web page may be modified after the annotation was added. In this case, the text fragment and the location of the fragment may have changed, but the annotation should still point to the same part of the text. This means that the location of the text in the dom may have changed and the fragment itself may have been modified.

To restore the annotation they use fuzzy matching in the following steps

  1. Identify regions that may hold the text
  2. Conduct a fuzzy search to find possible starting and ending points for the matching text
  3. Highlight the text that is the closest match from the candidates in the fuzzy search

The user highlights text in the original web page and the program stores the highlighted fragment along with text showing the context both before and after the fragment.

When the user loads the web page, the following steps are performed to locate the fragment

  1. Use jQuery body.text to extract all text from the web site
  2. Build a list of infrequently used words and locate these words in the web site text
  3. Use the JavaScript implementation of Google’s diff-match-patch library to find the fragment in the text (The library uses the Bitap algorithm to find the best fuzzy match). The algorithm finds starting locations for text matches to the fragment. If the fragment is longer than 64 characters, only the first 64 characters are used. Searches are conducted using the before-context with the fragment to determine the general location in the text and using only the fragment to determine the possible starting points of the fragment in the text.
  4. Reverse the order of characters in both the fragment and the text. Repeat the previous step to determine possible ending points of the fragment in the text.
  5. Extract candidate locations for the fragment and pick the location which has the minimum Levenshtein distance (fewest character substitutions/inserts/removals).
  6. Highlight the text in that location. Repeat this process for each stored fragment.

Next, Peter Brodsky @HyperScience spoke about how his company is making the training of neural nets more efficient. HyperScience trains neural nets (containing up to 6 layers) on a variety of tasks (e.g. looking for abnormal employee behavior, reassembling shredded documents, eliminating porn from web sites).

The problems they want to overcome are

  1. Local minimum solutions are obtained instead of a global minimum
  2. Expensive to train
  3. Poor reuse

To overcome these problems they do the following. Once the nets are trained, they examine the nets and extract subnets that have similar patterns of weights. They test whether these subnets are performing common functions by swapping subnets across neural networks. If the performance does not change then they assume that the subnets are performing a common task. Over time they create libraries of subnets.

They can then describe the internal structure of the net in terms of the functions of subnets instead of in terms of nodes. This improves their ability to understand the processing within the net.

This has several advantages.

  1. They can create larger and more complex networks
  2. They can start with a weight vector and guide the net away from local minima and toward the global minimum.
  3. Their networks should learn faster since the standard building blocks are already in place and do not need to be reinvented.

In the third presentation, Tarek Sherif @BioDigital talked about how BioDigital is implementing anatomical content for the web. The challenge is to create 3d, interactive pictures showing human bodies in motion or in sections, layers, etc.

BioDigital uses webGL to render their content in HTML/CSS/JS on all browsers and mobile devices. Due to the computational load, optimization of memory management and JavaScript code is important.

The content can be static, animated or a series of animations. The challenge is to keep the size down for quick downloads, but have the user experience the beauty of the images.

Displaying anatomical content is challenging since it can be

  1. Deeply nested – e.g. brain inside skull
  2. Hierarchical – is the click on the hand or the arm?
  3. Scale – from cells to the whole body

User interactions can include –highlighting, dissection, isolation, transparency, annotation, rotation,…

Mobile is even more challenging

  1. Limited memory and GPU
  2. Variety of devices
    1. GL variable limits
    2. Shader precision
    3. Available extensions

To allow their images to be plugged into web sites, they create an API

  1. Create an iframe to embed into a page
  2. Allows basic interactions
  3. The underlying JavaScript can be customized

API challenges

  1. 3d terminology and concepts
  2. 3d navigation
  3. Anatomical concepts
  4. Architecture of the Human

Examples can be seen at

The artists primarily use Maya and Zbrush as their creative tools.

Models can be customized for specific patients.

posted in:  Animation, applications, Code Driven NYC, Programming    / leave comments:   No comments yet