New York Tech Journal
Tech news from the Big Apple

#D3, #React and #Clustergrams

Posted on February 22nd, 2016


02/22/2016 @ Pivotal, 625 6th Ave, NY

20160222_192712[1] 20160222_194948[1] 20160222_200736[1] 20160222_200909[1]

Two speakers talked about combining the functionality of D3 + React and taking advantage of the power of D3 to create graphic displays for data analysis.

In the first presentation, Pan Wangperawong @_panw talked about how to integrate the power of D3 for graphics and React for dynamic updating of the DOM. Both D3 and React espouse functional programming and both want to manipulate the DOM, but only one can do this without conflict. React creates a virtual DOM to efficiently changes in the display. So, to utilize this facility, React should manipulate the DOM. This means that D3 does the math and creates objects that are passed to React, but D3 cannot directly access the DOM.

Pan illustrated this by walking through code behind an example application: The details are on Github.

An overview is that React creates DOM elements and takes responsibility for interactive manipulations. D3 is called within a single function to set up these elements. The D3 calls start in line 27 of his code.

In the second presentation, Nicolas Fernandez introduced clustergrams (heat maps with rows and columns of the matrix ordered by a cluster analysis) and talked about how D3 gave him the programming tools to create interactive displays of clustergrams and related data analysis pictures.

He talked about related methods for displaying data including force-directed graphs, adjacency matrices, and raw item by characteristic matrices. He illustrated each using with a matrix of characters and which chapters they appeared in Les Miserable. The matrix can be colored when a character (column) appears in a chapter (row). The number of overlapping chapter appearances can be converted to a similarity measure which can be plotted as a adjacency matrix, force-directed graph, etc.

For each of these, Nicolas demonstrated how an interactive graph helps us better understand the structure of the book.

Next he talked about how Python (using the SciPy tool) and D3 gives him the tools to make the interactive plots. Specifically, he talked about how D3 object constancy makes it possible to zoom in with the following effects:

  1. remove labels for rows that will fall off the end of the page
  2. reposition the remaining labels to cover the height of the page
  3. expand the matrix to vertically fill the page


He also illustrated other effects made easy to implement in D3.

His code is available on



posted in:  applications, d3.js, data analysis, Programming    / leave comments:   No comments yet

Responsible #DataVisualization & Becoming a #D3 ninja

Posted on November 23rd, 2015

11/23/2015 @ Pivotal Labs, 625 6th ave, NY

20151123_192319[1] 20151123_202423[1] 20151123_202635[1] 20151123_202805[1] 20151123_202930[1]

In the first presentation, Amanda Chung @DockYard spoke about Responsible Data Visualization.

She opened with a side illustrating a data representation from the art world:

Amanda described responsible design as one which does the following

  1. Form – Choose the right type of graph to match the purpose of the graphic: comparison , relationship, composition, distribution
  2. Integrity – pie chart should add to 100%, bubbles are accurately sized (size by area, not radius), don’t truncate axes – start axes at zero. See Darrell Huff’s book on “how to lie with statistics”
  3. Efficiency – readability, use legend, inline labels that appear when we hover. User should do as little work as possible
  4. Performance – (speed to load) – some css declaration are faster, consider transitions. See Laura Hogan book “Designing for Performance
  5. Accessibility – design the site to help people with disabilities (visual, motor, auditory, cognitive),
    1. Spectrum is a Chrome extension to show how a site would look like for someone with color blindness.
    2. Semantic HTML – input to screen readers
    3. Keyboard navigation – need signifier to tell you were you on the site (like gameboy). Also keyboard shortcuts
    4. Sharing a visualization – information in a ‘get’ can be emailed easily
    5. Responsiveness – hover rollover needs to be translated to mobile devices that have fewer pixels to use as a target
  6. Maintainability –
    1. flexible dimensions – responsive
    2. dynamic data– can you add more data points
    3. don’t repeat yourself – easy for scripting

SVG may have scalability problems for large numbers of data points. Instead use canvas

In the second presentation, Karl Sluis @Pandora’sNextBigSound spoke about “What I Wish I Knew Two Years Ago: A Designer’s Guide to D3”

He said that Scott Murray’s web pages were an excellent intro to D3

He also said that his slides are posted on @karlsluis on twitter.

Karl broke his advice into three areas.

  1. Tools –
    1. to avoid “XMLHttpRequest cannot load” – install node.js and npm install http-server –g so anywhere you can type ‘http-server’
    2. open the inspector to see how the code was written
    3. see
    4. see Mr.Data Converter – creates inputs to a JSON file
  2. Getting started – tips for D3
    1. <g> – what is that? = an SVG group – encapsulate when you can
    2. Data join
    3. $0.__data__ to see if the data is joined with the element = inspect the element
    4. Better to group items and then append the attributes
    5. Add a class name to make it easy to specify a group
    6. SVG starts top left. D3 margin convention – append it to a <g> and move to the left
    7. Write it out (an outline), then code it
  3. Data – (also great advise for any scripting language)
    1. Use a callback functions to get data
    2. Reformat data using a forEach loop
    3. Use functions
    4. Use console.log() to see what is going on
    5. Build slowly and incrementally

posted in:  d3.js, Programming, UX    / leave comments:   No comments yet

#Visualizing un-aggregated data using #D3.javascript

Posted on November 24th, 2014

NYC D3.js

George Murphy @georgeLmurphy @Nomi

11/24/2014 @Pivotal, 625 6th Ave, NY

Two presentations. One by George Murphy showing how he used d3 routines to create interactive web graphics. The second by Sebastian Gutierrez was a beginner tutorial on d3. My notes do not cover that presentation.

20141124_191818[1] 20141124_192103[1] 20141124_192535[1]

George presented four case studies

  1. Behavior timelines showed time lines for individual shoppers including the sequence and time spent on various activities in the store.
  2. Occupancy by hour showed when people are sitting at specific seats in a restaurant. The site showed the occupancy of seats on a floor plan and individuals hours can be clicked to show the percent occupancy and which seats are occupied at that time.
  3. Nomi, his current company, placed 250 large “eggs” at various points in Manhattan. Each egg had a beacon so smart phone users could “collect” eggs. George showed a site where one can query by location, accumulated visits and select eggs by common themes.
  4. A site for exploring the trick-by-trick elements in a head-to-head skateboard competition. Tricks were synced to videos and results could be viewed in many ways: by elimination round winners, tricks successful/failed by individual, round, etc. offensive/defense and keyed to the videos.

George mentioned the following useful resources on visualization

Bret Victor – Ladder of Abstraction

Books: “Visualize This” by Nathan Yao

Books – “Align Left” by Scott Murray

posted in:  applications, d3.js, JerseyScript    / leave comments:   No comments yet


Posted on October 28th, 2014

NYC D3.js meetup

10/27/2014 @Pivotal, 625 6th Ave, NY

Two presentations showed how #d3 is used to create #interactiveGraphics.


In the first presentation, Paul Levande, originally from #Westport, CT, showed an interactive tool he developed to display the presidential voting preferences of the 23 towns in Fairfield county CT. Taking town-by-town voting records since the 1924 elections, he created a two panel display. The left display was a scatter plot comparing the percentage of voters voting for the Democratic presidential candidate for any two elections. The years of the x and y axes were selected by dropdown menus. The right display was a line graph showing the percent voting Democrat from 1924 to 2012 with each town represented as a separate line. Data for each election and town appears when points are clicked and lines for each town can be highlighted by clicking on the line


The second presentation was presented by several speakers from @Objectlab, a boutique consulting firm. They used d3 to create a monitoring tool showing #Sony music’s #digitalSupplyChain. The #graph is a real time monitoring visualization with aggregated history – number of packages that are in the pipeline. Green is good and red shows a delay in the process. The right side shows production over the last 60 minutes at 1 minute intervals. The left side shows 1 and 2 hour aggregations. They also presented the code and described the backend architecture that served the data. The code is available at

posted in:  d3.js, data, Programming    / leave comments:   No comments yet