New York Tech Journal
Tech news from the Big Apple

How to build a #MixedReality experience for #Hololens

Posted on April 14th, 2017

#NYSoftwareEngineers, @NYSE

4/14/2017 @MicrosoftReactorAtGrandCentral, 335 Madison Ave, NY, 4th floor

Mike Pell and John gave a roadmap for generating #MixedReality content. They started with general rules for generating content and how these rules apply to building MR content.

  1. Know your audience –
    1. Role of emotion in design – we want to believe in what is shown in a hologram.
    2. Think situation – where am I? at home you are comfortable doing certain things, but there are different needs and different things you are comfortable in public
    3. Think spatially – different if you can walk around the object
    4. Think inclusive – widen your audience
  2. Know Your medium
    1. For now you look ridiculous when wearing a VR headset– but maybe this eventually becomes like a welder shield which you wear when you are doing something specialized
    2. Breakthrough experience – stagecraft – so one can see what the hololens user is seeing
  3. Know Your palette

Interactive Story Design – a fast way to generate MR content

  1. Character
    1. Who is your “spect-actor” (normally someone who observers – have a sense of who the individual is for this moment – avoid blind spot, so pick a specific person. )
    2. Who are your “interactors” – will change as a result of the interaction – can be objects, text, people
    3. This creates a story
  2. Location – design depends on where this occurs
  3. Journey – how does participant change

How to bring the idea to life: how to develop the script for the MR experience

3-step micro sprints – 3 to 6 minute segments – so you don’t get attached to something that doesn’t work. Set 1 to 2 minute time limit for each step

  1. Parameters – limited resources help creative development
    1. Personify everything including text has a POV, feelings, etc.
    2. 3 emotional responses – what is the emotional response of a chair when you sit in it?
      1. Negative
      2. Neutral
  1. 3 conduits
    1. Language
    2. Facial expression – everything has a face including interfaces and objects
  1. Playtest – do something with it
    1. 3 perspectives
      1. Participant
      2. Interacters – changes in personality over time
  1. PMI – evaluative process – write on index cards (not as a feedback session) so everyone shares their perspective. Next loop back to the parameters (step 1)
    1. Plus – this is interesting
    2. Minus – this weak
    3. Interesting – neither of the above “this is interesting”

How to envision and go fast:

  1. Filming on location – randomly take pictures – look for things that speak to you as creating an interesting experience.
  2. Understand the experience – look at the people (i.e. people viewing art)
  3. Visualize it – put people into the scene (vector silhouette in different poses) put artwork into scene along with viewers.
  4. Build a prototype using Unity. Put on the Hololens and see how it feels

They then went through a example session in which a child is inside looking at a T-Rex in the MOMA outdoor patio. The first building block was getting three emotional responses for the T-Rex:

  1. Positive – joy looking at a potential meal: the child
  2. Negative – too bad the glass barrier is here
  3. Neutral – let me look around to see what is around me

To see where we should be going, look at what children want to do with the technology

posted in:  Animation, Art, UI, video    / leave comments:   No comments yet

Building an #AI #AutonomousAgent using #SupervisedLearning with @DennisMortensen

Posted on March 23rd, 2017


03/21/2017 @ Rise, 43 West 23rd Street, NY, 2nd floor

In mid 2013, started to employ machine learning to set up meetings. After an introduction to the software, Dennis talked about the challenges for creating a conversational agent to act as your assistant setting up business meetings.

He talked about the 3 processes within the agent: NLU + reasoning + NLG

Natural Language Understanding needs to define the universe – what is it we can do and what is it that we cannot do and will not do?

Natural Language Understanding (NLU) Challenges

  1. Define intents then hire AI trainers. Need to get the intents right since it’s expensive to change to a different scheme
  2. What data set do we align to? What are the guidelines for labeling? Coders need to learn and remember the rules defining all the intents. Need to keep it compact, but not too much so
  3. They have 101 AI trainers full time. On what software do they label the words? Need a custom-built annotation platform. Spent 2 years building it.

Reasoning challenges

  1. How do people want the agent to behave? Manually determine what is supposed to happen. This will create a new intent, but this often requires changes in the coding of the NLU
  2. Some of the things humans want to do are very complicated. Especially common sense
    1. don’t do meeting after 6:00, but if there is one at 6:15, there is a reason for this happening.
    2. a 6:30 PM call to Singapore might be a good idea.
    3. When to have a meeting and when to have a phone call

Natural Language Generation (NLG) challenges

  1. They have 2 interaction designers
  2. Need to inject empathy if it’s appropriate. For instance if there is a change in schedule, we need to respond appropriately: understanding initially and more assertive if the change needs to be unchanged. Also need to honor requests to speak in a given language.

They evaluate the performance of the software when being used by a client by

  1. customer-centric metrics, such as the number of schedule changes
  2. is the customer happy?

posted in:  AI, Data science, NewYorkAI, UI    / leave comments:   No comments yet

#Neurobiology of #ComputerInterfaces

Posted on March 18th, 2017


3/17/2017 @Hunter College, 68th & Lexington Ave, New York, Lang Theater

Three talks were on the neurobiology of the computer interfaces were given by

Ellen Pearlman – Utopic or Dystopic

Ruben Van de Ven – Emotional Hero/We Know How You Feel

Greg Garvey – Split Brain

Ellen Pearlman talked about brain computer interfaces. The most available commercial devices are from Emotive, Muse, OpemBCI. The devices need smoothing and feature extraction algorithms to find signal in the noise. Devices take 8 seconds to calibrate and 150 milliseconds for a signal to be detected and transmitted by the device. One of the key signals is the P300 MERMER which indicates that you recognize someone/something.

She talked about the large increase in brain research funding recently by DARPA, NIH and NSF with the IARPA (Intelligence Advanced Research Projects) program soliciting proposals.

Ellen next talked about

  1. Semantic brain maps, which highlight the locations associated with particular stimuli
  2. Optogenetics which can implant false memories which can be turned on and off with blue and orange light
  3. Cortical modems, which can transmit images directly to the brain bypassing the senses (Cubic Corp is working on this)
  4. Brain data stored in the cloud (Cloudbrain and Qusp have been uploading brain data)

Finally, she showed her performance piece, Noor, a brain opera, in which a performer’s brain waves trigger visual patterns as the performer interacts with the audience


Next, Ruben Van de Ven  talked about the challenges faced by machine learning methods that claim to determine one’s emotions from facial images. Applications using this technology include ‘Emotion Hero’ a games you can download from the Google Play Store and Hire-vue which evaluations people during job interviews.

Paul Ekman developed the ‘Facial action coding system’ which is the classification scheme used. But, Ruben notes that context affects how we interpretation an expression. Also the validity of the face classification is reliant on human subjective coding in the 19th century from a French asylum. Both place the science behind these methods on shaky ground.

In addition, the software is often marketed as both objective and as a tool for training individuals to mislead the software. But how can the software work if one can learn to manipulate it?


Greg Garvey talked about how his art installation take advantage of the modularized brain which is split into left and right processing. His installations show different images to the left and right eyes (and therefore to the left and right brain hemispheres) to raise internal cognitive conflicts in the image being viewed.

posted in:  Art, psychology, UI    / leave comments:   No comments yet

#Web and #Mobile #Development Panel

Posted on March 2nd, 2017


03/02/2017 @TheYard, 106 West 32rd Street, NY 2nd floor

A panel consisting of the following people spoke about setting up your startup.


Founder & CEO, Torops, Konstantine Sukherman

Founder & President, Mango Concept, Michael daniels

Founder & Creative Director, Awesome, Firat Parlak

Managing Partner, New Logic Technology, Alex Sokoletsky

Founder & CEO, bromin7, Sergey Belov


Recommend platforms to create MVP website. WordPress, but depends on the client, how fast you want to get up and running, and who are the clients. Also some funding shops have custom platforms. Drupal & WordPress are good alternatives. A couple of days are all that is needed to create an MBP. The platform depends on product. WordPress is not a fully scalable application. Invest custom if your product needs to scale to be successful. 70% of web is WordPress. 1 million visits/day is often the point when scalability becomes a problem.

When does UI/UX become important? You should define the UX before you build. You will then do a better job of predicting your costs and features along with the time frame.  It’s the most important part of the startup. A prototype will make your pitch easier.

How do you build a dev team? Web sites: Drubal, Behance, WorkingButNotWorking. Initially reach out to your network. For a founder its different – what is the skill set are you are looking for?

How do you hire a CTO? Alternative is to outsource or get a technical advisor (a few hours/week). One of co-founders is better if they have technical background. If just starting, will need to offer CTO a lot of equity. If need technical co-founder might offer equal or even more equity. At later stage, will need to give less equity. Also the CTO might be good technically, but need not know the all the area of dev. Get people excited.

An internal team will give you greater control. But partnering might be most cost efficient. Want to build a long term relationship – outside tem must be interested in the product. In-house developers must have equity. Don’t squeeze dev too much – it’s about building a relationship. Everyone should be happy. Good dev are hard to find > $100k/year. Get it out in the market as quickly as possible. Need proof of concept in the market. Get people in house to manage the outsource developers. 40% in-house and 60% outside can be a good mix. Find a senior designer to start (don’t leave it to a junior designer).

How do you choose a dev shop? Needs to more than a dev shop – need business analysts. Startups have great ideas, but need a partner to help on strategy. Need technical knowledge but should also have interest in the field. Personalities need to match. Good idea to keep some people who built the project even when you are able to hire an in-house team. You need to build a communication channel. Select the shop or can build an off-shore team – depends on whether you need to senior staff. Need to interview the developers who will be working on the project. Can scale faster if you get the right partner.

Okay to the start without understanding the code. Need to get someone who will take over the task. Should talk to developers 2x-3x/day. This will help you generate ideas. Every good dev team will give you an estimate, but it’s just an estimate. Don’t try to push you agenda on your developers. If it takes longer, they are trying to make the product better. Get out as quickly as possible if dev team is not producing or communicating.

Will an angel invest in a company which is using a dev shop? The VC at some point will ask you to build an internal team. If large amount being raised you may be asked to put together an in-house team. Otherwise, you may do either. Investors are looking for a good idea! Be clear to the dev shop on the amount of money that is available to use for dev.

The technology stack is key if you are developing in house. If the CTO is using an outside team, then it is sufficient for the CTO to have a computer science degree and decent resume. Mainly need someone who can explain complex things to non-technical people. if AI is involved, the CTO should understand the core ideas of AI.

Never speak technical language to investors. Investors want to know the idea, revenue streams, the team.  Investors like to know how product will function, so they want to know on-boarding and scaling, but they don’t need to know the dev stack. When you want to raise millions, investors will do a tech review so they know if the product will scale.

Usually need some type of MVP, but it can be small and cheap to develop. Raise money for this from friends and family.

Make sure you understand the scope – detailed scope will help prevent dev overruns. If you need to go outside the scope, then cut back features. Break it down to multiple phases and emphasize the key features. Always have a 20% buffer. Design dictates the development. Lock in the design since changes for developers are expensive.

Any good dev shop will have a flexible contract, so you will need to pay for design changes, but they will need to absorb charges if they misestimate the time. Also it depends on the amount of overage and the relationship.

You cannot really protect the idea. Otherwise, patent it and you might be able to negotiate to let the shop share the tech in return for a lower price.  Don’t be afraid that someone will steal your idea. Executing is the difficult part.


posted in:  startup, technology, UI, UX    / leave comments:   No comments yet

DesignDrivenNYC: growing the design group and design function in organizations

Posted on May 11th, 2016


05/10/2016 @WeWork, 115 West 18th Street, 2nd floor, NY

20160510_182213[1] 20160510_184933[1] 20160510_185328[1] 20160510_192333[1]

Three speakers spoke about their methods to integrate #design processes into all parts of product #development within retail-facing #financial firms. Betterment is a small, but rapidly growing company that is upsizing the design team. Citi and Learnvest are moving from a haphazard view of design to a single customer-centric design process throughout the organization:

In the first presentation, Jamie Strollo @Betterment spoke about the challenges the UI/UX design team faces as the company goes from a small startup to 9 designers to doubling by year end. Originally Betterment had a flat organization, but now there are challenges: integration of new people and avoiding the bad dynamics of large meetings. Design is the only shared resource across the company, so there was duplication in work. Some strategies for tackling a new design challenge are:

  1. Kick off strong – ask what is the problem?, what is success?, how do we measure?, constraints? Initially concentrate on measuring the drop-off rate when evaluating a design change. But eventually shift to measuring the effect on profitability.
  2. Assumption gathering – for stakeholders, high-level activity, focus on fears and confidence, agree of riskiest assumptions, talk to customers to validate.
  3. Focus on Top 5’s
    1. Great for large groups, Iterations
    2. High-level activity
    3. Select only top 5 design needs
    4. Helps to Establish patterns
    5. Bridge the “delete” conversation
  4. Ideation / paper prototyping – better for smaller groups. Bring in other areas of expertise, let others have a voice
  5. Managing feedback – decide who are the decision makers, who to inform. No big UI critiques (a polished presentation gives a finished feeling and makes it hard to change), share often and early, speak about objectives and key results. Start conversation by what is the objective.
  6. Invest in relationships – customers and coworkers

Another challenge as the company grows is creating a method to give better estimates of the time to complete a design. This is hard since much of time goes into understanding the problem

In the second presentation, Billy Seabrook @Citi spoke about how Citi has created a single world-wide design team. The goal of better design is to move customer’s view of the bank from a transactional experience to a relational experience.

Starting six months ago, Billy has approached the following challenges within the bank:

  1. Organization – create agile groups adding individuals looking at strategies; research and usability studies; producers to keep on time and budget.
  2. Skills – Create a cohort of coaches to teach design thinking throughout the bank. Minimal viable product is at the intersection of business viability + customer desirability + technical feasibility; Partner with IDO to foster agile design thinking throughout all parts of the bank
  3. Applied Projects – Citi Fintech launched 6 months ago to launch the bank of the future: focus on mobile (mobile only), speed and simplicity (2 weeks of design thinking + 2 week dev sprints)

To foster common branding and look-and-feel world-wide, document templates are shared world-wide. Senior people in each location are in constant contact.

Coaches have backgrounds of policy or planning. The main thing is the mind set of being customer centric.  In the past, most of the product design was outsourced, so design principals were inconsistent also there was a lack of consistency in evaluating designs.

The Design group reports to the COO of Fintech and is considered a cost center (despite its’ close affiliation with profit centers).

In the final presentation, Abigail Hart Gray @Learnvest (help financial planners create simple, effective, .., plans for retirement…) talked about the challenges of integrating a unified design process into product development at Northwestern Mutual (acquired Learnvest last year).

Abigail started with the question of a Defining Design Driven? She interviewed experienced design professionals and found commonalities:

  1. Team structure – be at the table when decision are made
  2. Process – iterative process
  3. Outcomes – but interviewees disagreed up whether good design resulted in products that were best for customer or best for business.

She talked about becoming a champion of design within the company:

  1. know the capabilities
  2. need to invest in design
  3. designers must educate their audience and sell their vision.

If you need to explain the interface, it stinks!

Steps to get started (data-driven, customer centric, outputs oriented):

  1. pick something with low stakes – no bottom line implications
  2. research well
  3. measure everything
  4. share results
  5. repeat

As an aside on the measurement process, one needs to consider the possibility that customer behavior changes by knowing that they are being observed. The Hawthorne effect can elevate or suppress responses depending on prior customer engagement (friends&family vs. the general population), frequency of engagement (daily vs. occasional), etc.

posted in:  applications, DesignDrivenNYC, finance, UI, UX    / leave comments:   No comments yet

The Rise of the #DataArtist

Posted on March 9th, 2016


03/09/2016 @ PivotalLabs, 625 6th ave, NY

Olivier Meyer & Ryan Haber@Zoomdata talked about the advantages of interactive #DataAnalysis. They showed how a single picture can show the ruin of an army through cold and casualties. This was done by Charles Minard in his graphic of Napoleon’s 1812 invasion of Russia. There, 6 time series are displayed to great effect.

Next, they talked about the complexity of displaying facts buried in large data sets. This complexity creates a new category: Data artist who sits between the business analyst and the data scientist

They demonstrated how their program facilitates the interactive search for patterns in the data by retrieving only the relevant subset when needed for the graphics display. They call this microservices & data sharpening (initially a rough picture is presented, but results are refined as you watch).

Many interesting points were brought up in the discussion.

  1. Before diving into the data, one needs hypotheses of what is relevant to decision making
  2. Care must be taken, since interactive graphics (as in all graphics – see Darrell Huff “How to Lie with Statistics”) can inspire misleading or unfounded conclusions
  3. The data artist is obligated to present graphics that are truthful
  4. Generic templates may not be the best data presentation
  5. One needs to balance the customization of the data presentation with the time & effort expended to create an improved graphic
  6. Graphically inspired conclusions need to be supported by relevant statistics
  7. Frequently, statistics (alone) are not the best way to present findings
  8. The best way to communicate is dependent on the audience.
  9. The tools for data exploration may or may not be different from those for presenting conclusions.

posted in:  UI, UX, UX+Data    / leave comments:   No comments yet

Mobile Display of #NewJerseyTransit #Bus and #Train Updates

Posted on January 14th, 2016


01/14/2016 @ 50 Harrison St, Hoboken, NJ

20160114_193042[1] 20160114_193226[1] 20160114_193643[1]

Bob James @ NJTransit spoke about their acquisition and dissemination of real-time train and bus data. This is exemplified by the current bus locations shown on

Bob talked about 12 years of automating the collection and reporting of real-time data. The initial goal was to automate arrival and departure announcements at train stations. Doing this required locations of trains and buses in real time. This eventually led to DepartureVision and other in-station and mobile displays of arrival times. (Locations with LED signs will eventually have countdowns included in the display.) Systems are also being upgraded from recordings (female voice) to text-to-speech systems (currently a male voice) by Loquendo. In most cases, the triggering of announcements is fully automated.

Ticket sales were the first items in the mobile app. This was followed by trip planning, station information, etc.

Buses. Currently all 4,000 buses are tracked by GPS. The MyBus app shows bus locations in real time. A system was also developed to record when buses leave, number of passengers, departure gate, … This is populated by a central database and updated using mobile apps. Within NJT, eStarter, a mobile app reporting arrivals and departures at Port Authority, is used by dispatchers to reschedule and redirect buses to keep them on time. The system replaces manually created monthly performance reports. It also has proven useful during storms when counts of morning commuters are used to estimate the evening demand. Clever Device automates the counting of people boarding and exiting each bus.

Trains. The position of each train is reported by track circuits since trains do not have GPS on board. 50 stations have wifi provided by CableVision. Spring 2016 will see the rollout of onboard wifi. Currently, most mobile access is through 3rd party apps which can combine data from other transit providers. Eventually he sees a move from the current feeder system to Uber. This will also make it easier for all riders to get to and from the train station.

posted in:  applications, MobileDevNJ, UI    / leave comments:   1 comment

Design Driven NYC: #TypeFace, an online space for #Designers, #Design for non-designers and designers

Posted on December 9th, 2015


12/8/2015 @ TheNewSchool, 66 West 12th Street, NY

20151208_182126[1] 20151208_182258[1] 20151208_182813[1] 20151208_185410[1] 20151208_192352[1] 20151208_194057[1]

Four speakers talked about design related concepts:

In the first presentation, Gale Anderson @SchoolOfVisualArts shared her knowledge and enthusiasm for different type sets. Gale talked about various design projects from her largest, a sign on the campus of the Pennsylvania College of Art and Design in Lancaster, PA to her smallest, a U.S. stamp commemorating the emancipation proclamation.

She as co-authored with Steve Heller 12 books typefaces and wrote the book “Outside the Box” on hand drawn type on packages.

Gale also talked about how her passion for type extends to a collection of bottle caps and pictures of outdoor signs in NYC.

Gale spoke about the many authors who have inspired her and given her insight. These include

  1. Paul Rand and Alex Steinweiss
  2. Alvin Lustig
  3. Beverly Thompson
  4. Bruna Munari
  5. Saul Bass

Some of her favorite contemporary designers include

  1. Fred Woodward
  2. Paula Share
  3. Louise Fili – signs in Italy & Paris

She recommended as a reference book – Ellen Lupton “thinking with type”

The second speaker talked about a site for posting your accomplishments in the world of design. Anil Dash @Makerbase has created a site where designers can memorialize their work and seek like minded people.

Anil talked about the difficulty in getting credit for your accomplishments and how the site strives to make the site a good environment. He talked about discouraging “drive-by contributors” with the following rules

  1. Persistent identities – so you can see the history of what you have done. Don’t require real names, but you cannot be anonymous
  2. Adding accountability around content
    1. Whenever you change things – the editing actions are visible in context.
    2. Show your actions to your peers – receive email notification.
    3. Trust people to use their judgment.
    4. You can edit your own page.
    5. All your actions are viewable in your profile.
    6. There is a moderator for content who will get an alert if there is an “edit battle” (if a page is being modified frequently by two individuals) or other disturbances
    7. Make it easy to flag anything, anywhere
    8. Make a place people can trust
    9. Anything you put on the site is public.

He said the biggest challenge is to get people to come back and also for people evaluate what other people are doing.

He wants to make the site a place to share information on how to solve problems.

Next, Sara J Chipps @Jewelbots talked about how she, as a non-designer, manages designers.

Sara loves good design and spoke of how important design is in the products (Amazon Echo, Glossier , Bustle, Caeden (headphones)) she uses and the games (monument valley) she plays. However, she is not a designer. She gave recommendations on how to acquire good design if you are not a designer and how to communicate with designers.

Sara’s Principles (for those who are not good at design):

  1. Copy those who know how to design – Googled “beautiful slide deck”: Solid color background, sans serif, mostly in caps, etc.
  2. Illustrator and photoshop are difficult. Like pixelmator since it is easier to use.


When dealing with designers in your business

  1. Hire for perfection – and passion for the craft
  2. Never be prescriptive – better to share what your emotional response when you view the page.

Finally, Josh Long @Oscar (health coverage) talked about how design fits into a rapidly growing company that needs to design for many groups: patients, doctors, etc.

He talked about ways to make design work

  1. Don’t wait for things to happen – come up with your own objectives.
  2. Everyone has the right to come up with ideas, so respect what other are creating
  3. Experts need to make the big decisions – it’s a balancing act
  4. Don’t be precious about anything – your first idea might not be the best

Embrace the chaos.

When asked about the type face for the whimsical ads on the subway, Josh said that they wanted to balance the playful nature of the graphics with serifs on the type face to emphasize the serious nature of the product.

posted in:  DesignDrivenNYC, UI, UX    / leave comments:   No comments yet

CodeDrivenNYC: caching web pages, #NLP, bringing #coding to the masses

Posted on November 20th, 2015


11/19/2015 @FirstMark, 100 Fifth Ave, NY

20151119_182753[1] 20151119_184814[1] 20151119_185517[1] 20151119_190304[1] 20151119_191249[1]

The first of the three presenters, David Mauro @Buzzfeed spoke about creating Mattress, their first open source IoS framework. Mattress caches web pages for later, off-line consumption. It also makes it appear that the page is loading quicker when online.

David spoke about the hurdles implementing this product

  1. How do we download an entire web page?
  2. How do we provide the content back to user

Their first decision was to download the URL using UIWebView and then capture all requests as they come through using  NSURLProtocol. UIWebView runs on main thread and is resource intensive, but the alternative to manually parse the HTML and the JS. They download the URL using UIWebView and then capture all requests as they come through using  NSURLProtocol. But WKWebView does not handle NSUIRLProtocol and there is a bug so you cannot just save another NSURLCache. They use commonCrypto to retain the URL, with the name hashed so even the longest name is uniquely identified.

They also need to know when a page if done downloaded.  Automated solutions have tendencies to either terminate prematurely or not terminate at all. Instead, they ask the user when the download is done.

How to provide the content back to NSURLProtocol? First ask the user if they are offline. If so, they retrieve the page from the custom offline cache. If they are online, the system reloads the initial request.

The system was designed as a simple API that can be run either in foreground or in background fetch. The background fetch needs to be monitored carefully so it does not use too much of the battery or slow the processing excessively.

The second speaker, Rob Spectre @Twilio demonstrated how easily applications can be made interactive using the Natural Language Processing tool, Textblob running in python.

Rob showed how to create an app that receives SMS text messages and changes its response based on your message. In just a few lines of code, Rob showed how the response can be differentiated based on the length of the message, it’s sentiment, it’s sentence structure, etc.

Ryan Bubinski @Codecademy asked the question “What is code?”

As an overview of the many ways to answer that question he recommended the 38,000 word article written by Paul Ford in Bloomberg June 2015

He summarized his view by saying that code is a lever that is becoming more powerful every day. As an example, he mentioned OpenFace, an open source program which uses a neural net for face recognition.

Making this lever available to more people requires either

  1. Making coding easier or
  2. Making it easier to learn how to code


posted in:  Code Driven NYC, iOS, Natural User Interface, Open source, Programming, UI    / leave comments:   No comments yet

What you get from unmoderated and moderated #testing of #prototypes

Posted on October 22nd, 2015

Jersey City Technology Startups

10/22/2015 @Ishi Systems,  185 Hudson St, Plaza 5, STE 1400, Jersey City , NJ


Steven Cohen @Validately spoke about the goals and advantages of prototype testing.

He started by reviewing his history in startups: one successful start and one failure. After these experiences he asked why does innovation fail?

  1. False positives – estimated that 50% of a typical product’s features are not used (see

He then asked what you can learn from users

  1. UI usability – can user complete a task by clicking on the appropriate links? – the inventor is too close to the product to know this
  2. Real life usability – get a lot of false positives here. “It’s not a big enough problem.”
    1. Are there blockers – usable in real life, such as privacy/security issues.
    2. Better alternatives – something I’m using now. What is it like?
    3. Does it solve a need? Needs to be big to change inertia. Need to get to “no” to show you know the issues. For instance introduce a price and ask if they would buy it (time, reputation, money)

Steven then presented two ways to test prototypes

  1. Unmoderated – asynchronous testing as the user explores the functionality – good for UI usability
    1. Less work, subjects pace themselves and give verbal/written feedback
    2. Can’t learn real life issues
  2. Moderated – live test interacting with the user and asking for opinions – better for real life usability
    1. Deep user learning – usability
    2. More time commitment

Steven did a simulated interview to show how the moderator would probe the user his/her actual usage of the product. One of the most important aspects is to get the user to make decisions on feature tradeoffs. The main tradeoff is whether a feature is work a specific amount of money and if not, why not and would there be a lower price point.

He wants to put up road blocks and see how people react – e.g.  put up a page and ask for a credit card (even if you don’t keep the number) – see how much commitment. This is an attempt to get the feedback that users might not way to give you since they do not want to hurt your feelings or be confrontational.

Other observations were

  1. If you need to pay your own existing customers to participate in a test, then the problem is not that important. Power users will probably always test for free.
  2. Validately will start video recording the user, but does not believe eye tracking adds much since mouse movements correlate highly with eye movements
  3. For unmoderated tests, if 5 of 6 users can do it. it’s probably adequate
  4. For moderated test, start with 6 to 10 people. If there is a consensus, you can get by with fewer. If there is no consensus, then your persona (a straw man demo) is probably not well defined.
  5. Once you have made a change, go back to the initial set of users and see if it fixes the problem – did we understand the comments?

The bottom line is, don’t build stuff that users don’t need

posted in:  Jersey City Technology Startups, psychology, startup, UI, UX    / leave comments:   No comments yet