#AR / #VR and the sense of place, AR #documentaries
Posted on March 18th, 2017
3/17/2017 @Hunter College, 68th & Lexington Ave, New York, Lang Theater
Four talks were on the use of AR/VR were given by
Samara Smith & Sarah Wright – Revealing Here: Using AR and VR to Transform Sense of Place
Ed Johnston – Augmented Asbury Park
Sam Topiary – Exit Zero: San Francisco Freeway to the Future
Samara Smith spoke about three installations and the role of the documentary
- Commotion for the #QueensMuseum in NY – tablet allows users to superimpose routes to work for individual commuters overlaying the physical 3-d display of the panorama of New York.
- Walking tour of #HambergerSquare Park in Greensboro, NC. Here walkers take a picture of historical objects in the park and are told the story of the park.
- Walking tour of Central Park in NY which uses GPS to activate sounds relevant to the location in the park.
Next, Sarah Nelson Wright spoke about VR projects to give people new perspectives on specific locations
- #HiddenVistas showed #HuntersPointSouth as a wild space in NY after industry in the early 20th century abandoned the location and prior to new construction there.
- An installation at the Queens museum used a VR mask shaped as a pair of binoculars to give users the impression that they were looking at distant scenes from within the museum
- A walking tour of #Soho called #InvisibleSeams superimposed images over advertising billboards of the conditions of garment workers and the pollution from garment product.
Ed Johnson talked about #AugmentedAshburyPark, an AR app which superimposed lost buildings/items on the #AsburyPark, NJ boardwalk. Using images from postcards, a visual and geolocation indexed interactive map was created from the carousel, the wreck of the Moro, etc. The app can also display these images by scanning historical posters from Asbury Park.
They are currently exploring use of Argonjs to revive the geolocation features of the app.
Finally, Sam Topiary talked about the challenges of documenting the history of the Hayes Valley, a site in downtown San Francisco. The challenge is how to tell the story of a part of the city that has seen transformations over time and whose transformations are indicative of the changing fortunes of the city and the inhabitants near the specific location. She concentrated on specific periods
- Grass roots organizations that blocked and eventually remove freeways through San Francisco
- Temporary conversion of the site to an urban farm for a 3 year period
- Construction on the site of luxury apartments as the neighborhood gentrifies
The periods correspond to the ups and downs of the local economy.
She asked the questions of how to communicate the nuances of the history and location without overwhelming the viewer.
She and other panel members talked about how to make an immersive experience but not make it so immersive that the audience becomes passive. Where does narrative fit into this continuum? How does one find and appropriate level of audience interaction?
#Holograms, #VR, #Technology for Kids, #HomeSecurity
Posted on February 15th, 2017
02/15/2017 @ Wework Chelsea, 115 West 18th Street, NY, 4th floor
The speakers were
- Jon Troutman, Founder and Chief Creative Officer at Canary (all-in-one smart home security system)
- Bethany Koby, Founder and CEO at Technology Will Save Us (DIY technology kits for kids)
- Sophia Dominguez, Founder and CEO at SVRF (the first search and discovery engine for VR content) and Founder at AllThingsVR (weekly curated newsletter highlighting top VR news)
- David Rose, Founder and CEO at PRSONAS (interactive self-service holograms)
The first speaker, David @ PRSONAS spoke about their product which is a hologram persona that can serve as a greeter at retail stores: provide product information, financial guidance, intake of medical symptoms, etc. The greeter is a flat holographic image of a person in what David called 2 ½-d display.
The software behind the hologram can provide appropriate hand gestures, show videos, instruct users to input data on a tablet, etc.
David talked about how they have customized the image to avoid falling into the uncanny valley (close to human-looking so feels creepy) by modeling the image as a non-human character.
Next, Sophia @ SVFR spoke about how her company is striving to become the common site for distribution of VR, #AR and #MR videos. She likened today for VR as the early 1990’s were for Yahoo, when distribution of web content was still in its infancy.
She talked about the barriers to widespread VR production. These include lack of universally available hardware to record VR, lack of editing tools, but most importantly, we don’t yet know how to tell a story taking advantage of the VR experience.
Next, Bethany @TechnologyWillSaveUs spoke about how her company is creating kits for students to experiment in creating their own technology. The kits contain sensors, motors, etc. and are linked to a programming language on their web portal which is an extension of Scratch.
As an example she demonstrated a programmable wrist band that can react to motion, etc.
Bethany then talked about their company strategy which emphasizes a range of products.
- Create a range of products: variety of prices, can create bundled products
- Product-market-fit: hardware is more difficult, so put development is on a tight production schedules with lots of feedback. Monitor ROI for various products.
- By having a range of product, there are activities for all parts of the company at any given time.
She talked about how the company strives to stay ahead of the competitions (Little Bits, Lego Mindstorms) by carefully target price points and creating a wide range of products for different age groups.
Finally, John @Canary talked about their stand-alone, in-home security system which is connected to an app on your phone.
He emphasized the importance of Product design = Relationship design
- Quality time – the app needs to interactive. They made it easier to access the time line of videos taken by the system
- Crisis management – can contact the police if there is a notification – help the homeowner overcome a crisis: the assist the home-owner filing an insurance claim.
- Trust – connected-home customers are concerned about privacy. Use ICASlabs recently released device security protocols
- A little magic – surprise and delight. Good example is Netflix onboarding that asks for your movie preferences then starts recommending movies upon the first use.
John also mentioned that they store videos of arrivals and departures, temperature, air quality, how active are occupants. Videos are stored 24 hours or 1 month depending on the contract. They are partnering with insurance companies to get homeowner discounts for using Canary.
HardwiredNYC: #VR, #3-dPrinting, #Hoverboard lessons
Posted on March 8th, 2016
03/08/2016 @WeWork, 115 West 18th St., NY
The four speakers were
- Jens Christensen, Founder and CEO of Jaunt VR (the future of cinematic VR)
- Cyril Ebersweiler, Founder and Managing Director of HAX (full-stack accelerator for hardware startups)
- Yuval Boger, CEO of Sensics (open source VR head mounted displays)
- Jonathan Schwartz, Founder and CPO of VoodooManufacturing (mass 3D printing manufacturing services)
Jonathan Schwartz @VoodooManufacturing talked about how his company fills a niche between very small batches to products and mass scale production. This is when one printer is too slow and injection molded parts are too slow to setup or too expensive for a small run.
Their factory in Williamsburg, Brooklyn has 125 desktop printers that can produce moderate volumes quickly. They do plastic printing and are not a competitor of Shapeways which concentrates on small batches of high end product.
Next, Yuval Boger@Sensics spoke about their open source (OSVR), middleware product which provides a single interface for a wide range of HMDs (head mounted display) and input devices (pointers, buttons, …) and programming software (such as Unity)
The OSVR is multiplatform, high-performance rendering utilities, highly extensible, open source
Cyril @HAX talked about the lessons learned from the explosion of interest in hoverboards. HAX was started 4 years ago in Shenzhen China as a hardware startup accelerator. They now also have offices in SF and NY and have funded 133 startups. Of these
- 60% B2C, 40% B2B – lifestyle, health, robotics, IoT (manufacturing)
- 60% in Americas 20% Europe, 20% Asia
Cyril talked about 8 lessons from the hoverboard craze
- The market is faster than our language – the market has already fragmented – you will only get part of the market.
- Commoditization has reached light speed. Protect yourself with science, software or a community
- Inventing is only half the battle – protecting and commercializing is just as hard (Segway was too early and too costly)
- Public domain invention – hoverboards benefit from the lack of patent in China’s “network model” of IP – profits area spread around with more minds competing to improve the produce or manufacturing process.
- Shenzhen’s supply chain power. The 5000 factories are very flexible in what they produce. Last year they were making tablets and selfie sticks. Now they are building hoverboards.
- Hits are hard to predict – in Jan 2015 at TechCrunch, there wasn’t much interest in hoverboards, but now they are a hit with an 80% drop in prices.
- Buyers beware – customers should do proper due diligence. Address? Fake components?
- The game is still on – 2016 will see a new set of winners and hoverboards will morph into other products
Cyril said that much of the low hanging fruit for B2C has been created and there is a lot of competition there. HAX concentrates more in B2B and emphasizes extremely early stage funding. Creating products quickly is one of the reasons they are located in Shenzhen.
He feels that Kickstarter prematurely gives away the initial idea, thereby making it harder to keep your advantage when creating/marketing the 2nd generation product which generates the actual profits.
Lastly, Jens Christensen was interviewed by Matt Turck. Jaunt VR is developed hardware and software tools for Cinematic VR. Jens received a Ph.D. in computer science before founding three companies: software middleware, swap books online, semantic search (sold to flipboard).
Jaunt works with Disney and other premium content providers. Even though they make a high end VR camera, they concentrate on creating software that makes creation of VR movies simple (an automated platform to create content).
They produce a high end camera since goPro needed extensive modifications to even get to a moderate level of acceptability (additional heat sink, storage, etc.)
They see the bigger challenge and opportunity in software: Stitching images together seamlessly continues to challenge. The need to guess at the depth makes it hard to create an automated rending solution. But, standard editing tools can be adapted and an experienced cameraman can be trained in a day to use the equipment.
The industry as a whole is still learning how to tell a story with VR as “there is no front of the camera”. Multiple points of action or special effects may need to be used to take advantage of the medium.
Other observations are
- Long term – subscription model when they have enough content (Netflix model)
- Most VR content will be accessible through smart phones, but release of high end headsets will generate awareness.
- Eventual users will wear light weight glasses that are driven off the phone. Also envisions use of see-through headsets.
- Currently people are comfortable watching for 10 to 15 minutes, but this limitation is primarily due to the obtrusive nature of headsets.
- Expects to see future generations of smart phones designed for VR.
- VR also needs to incorporate the social aspects of viewing. Facilitating conversations with friends would be helpful.
- For the immediate future he does not see VR impacting viewing habits for TV, movies, etc.
Hardwired: #VirtualReality Devices and #Drones
Posted on January 12th, 2016
01/12/2016 @ WeWork, 115 W 18th St, NY
Four companies talked about virtual reality devices and flying drones
• Nicholas Horbaczewski and Ryan Gury, CEO and Director of Product at The Drone Racing League (drone racing sports league)
• Amir Rubin, Founder and CEO of Paracosm (cloud-based 3D mapping)
• Andre Lorenceau, Founder & CEO of LiveLike (VR sports broadcasting)
• Jan Goetgeluk, Founder and CEO of Virtuix (immersive virtual reality system)
In the first presentation, Andre Lorenceau & Jeremie Lasnier @Livelike spoke about developing a system so sports fans at home can view the game as if they were in the stadium. Their system will eventually allow fans at home to use a virtual reality headset to look around the space and do on-demand access of other views of the game.
Their system is designed to use current in-stadium video streams (the ability to see different parts of the playing field will initially be based on a single feed from a wide-angle camera at midfield), but be upgradable as new in-stadium, streaming technology is rolled out. (This differentiates them from NextVR which is building a streaming platform from the ground-up)
Andre and Jeremie talked about the challenges in getting VR right. One of the challenges is lack of a standard controller with different controllers specifying different gestures for similar functions. Interacting with a touch pad may not be correct in some circumstances, while pressing a virtual touchpad may be hard to do. Monitoring one’s gaze may work in some cases, but they need to avoid unwanted changes in the video when starring at parts of the action.
On the up side, there are many new opportunities to enhance as well as monetize the experience. Electronic placement of ads and user services are a possibility. These could be similar to the lines superimposed on the field during football broadcasts or they could be virtual objects moving through the space (they showed a video in which a Star Wars ship flew over the playing field).
In the second presentation, Nicholas Horbaczewski and Ryan Gury @DroneRacingLeague spoke about their soon-to-launched drone racing series. In these competitions, professional drone pilots will fly standardized drones over a race course. The fliers will wear goggles showing a video feed from the drone as the drones fly at over 80 miles per hour. The course will take less than 2 minutes to fly with winners determined by their performance over a series of heats.
Nicholas and Ryan spoke about their tuneup races held in Yonkers and in Sun Life Stadium in Miami. They prepare 80 to 100 rigs prior to the competition with all rigs made with the same hardware, but tuned to the liking of each competitor. They talked about five keys to a successful race
- Performance – drones are constructed from the highest quality parts with multiple identical rigs provided to each competitor for the four day event – includes practice, preliminary heats, finals
- Visibility – the sport is a spectator sport so the drones need to be visible from the ground. To do this, each drone is covered with high intensity lights with each participant identified by a different color.
- Pit crew – drones must be able to withstand crashes with minimal damage, so they can be repaired quickly. Everything needs to be tested.
- Stability – Due to the number of drones needed, each needs to be assembled and tested quickly. This necessitates maximal use of circuit boards and minimal wiring. Drones also have a large number of customizable settings.
- Radios – use analog video feeds to the pilots. (digital video has latency and also drops frames) they need to install a robust network for communications throughout the course even as it snakes through tunnels and around obstacles.
Returning to VR, Amir Rubin @Paracosm talked about the software his company produces to take a point cloud extracted from the physical world and create a picture of the surfaces that can be used to create a virtual world.
He first talked about why an accurate picture of the world is needed to create a truly immersive game to insure the following are true of the experience.
- Need correct perspective & occlusion
- World locking – persistence in physical world, no jitter
- interactions – shadows, physics, path planning
to solve these issues one needs to
- know shape and geometry of the world –
- know where I am in the world
Paracosm takes coordinates of objects extracted from the world using remote sensing devices such as the Kinect or RealSense (Project Tango) and sends them offline for processing to create a model of a space detailing walls furniture and other interior content. The offline processing removes the extra mesh points that explode memory and slow down processing (for instance the Project
Tango by itself cannot retain all the points is sees so it culls them, thereby limiting the scope of any game played on it). The refined set of points is then returned to the device (in this case the Project Tango) to serve as the framework for the augmented reality world.
Once this is done, the device can create characters that appear to interact correctly with the real world or superimpose data or views on the real world.
This offline processing makes a more true-to-life augmented reality game, but cannot react quickly if objects move in the space. This means that consumer product is still in the future. For this reason, Paracosm is concentrating on commercial engineering applications which would only involve static items.
In the final presentation, Jan Goetgeluk @ Virtuix talked about the Omni, which his company produces. The Omni allows you to walk around in the virtual world in a device that looks like a toddler’s walker. The device holds you in place, but allows you to walk in any direction (an omni-directional treadmill). The device will be sold starting this month for $699 and has no moving parts to track you footsteps.
Jan talked about his personal journey from Belgium, to the U.S., to Rice University, to investment banking, to entrepreneur. He talked about how it has taken him 8 years to come to production including three rounds of fund raising totaling 8mm USD. He talked about the delays and challenges even for a product that had early enthusiastic support from the gaming community, Kickstarter and Shark Tank.
NYVR: designing a #VR app and making the VR user comfortable with the experience
Posted on September 18th, 2015
09/17/2015 @ Microsoft,11 Times Square, NY
Martin Schubert talked about his award winning entry in the 2014 3d Jam and Eric Greenbaum talked about making VR pleasant for humans.
After a brief introduction to the technology of the #LeapMotion (two infrared LEDs that produce a black and white image from which one’s hand and finger positions and gestures are determined), Anthony introduced Martin Schubert who created the VR program Weightless (youtube video) using blender, unity, and playmaker. Martin described his process to create the app in 6 steps
- Identify the strengths of VR – 3d depth and sense of scale; easy to look around; good spatial awareness; sense of depth in a mid range around 2 meters
- Identify the strengths of the Leap Motion – hand motions are natural 3d inputs; display of hands creates body presence; weak in precision pointing (binary inputs); likes fingertip interactions, but there is not haptic feedback -> as a result, moving objects in a weightless environment was more natural that in the presence of gravity (there is mass, but we don’t need to fight against weight)
- Create prototype
- Create a narrative. Sorting objects in a space station (weightless environment). Have environment set the scene and create user expectations
- Repeatable actions. Get objects, sort, repeat
- Create a believable space – create points of interest. Set up the user initially (see video). Need to identity what is important. Have as many things as possible react to you
Marin also talked about taking advantage of the widgets in unity. He also said that is it important to have differentiate the foreground from the background and music should be part of the active space and interact with actions
As an aside, Aboard the Looking Glass won first place in the 2014 3D Jam
In the second presentation, Eric Greenbaum talked about considerations when making VR that does not make the user sick.
The key concept is presence so that the user forgets that technology is mediating the experience.
Some considerations are based on hardware: Tracking with low latency and low persistence. 1k by 1k per eye is sufficient resolution. Good optics
But, there are also human physiological considerations:
We are evolutionary primed to avoid experiences that made us nauseous in the past.
- Our bodies strive to match signals in the inner ear with what we see.
- Give users control of movement
- Avoid acceleration and deceleration – Trick is do instantaneous acceleration
- Keep things on the level plane
- Ground users with fixed objects a cockpit is one way
- Keep horizon steady
- Keep objects in a comfortable space – 6 to 10 feet is best
- Avoid things that fly at your eyes.
- Sound is important
- Design environment – People are afraid of small enclosures, high places.
- Sense of scale is important
- Interaction design. Text is difficult in VR. Guiding light or sound is helpful
Different design considerations for mobile and for desktop
Hardwired: development steps, #RealSense from Intel, the #3dPrinting business
Posted on September 16th, 2015
09/15/2015 @WeWork, 115 West 18th Street, NY
Four speakers spoke about hardware products from cutting edge sensors to lessons learned developing hardware products.
The First speaker, Saar Yoskovitz @Augury talked about his company’s experience in lean hardware development. Augury was founded four years ago and produces vibration sensors tethered to a smart phone that analyze the mechanical performance of HVAC systems. Saar described their design and development process.
He started by noting that the technology is not that interesting in most products. The key questions to ask are
- What is the product?
- Who will buy it?
- Why will they pay for it?
He advocated an agile development process to get feedback for improvements as quickly as possible: fast iterations. Min effort to get max learning.
- Research : get out of the building, stick to the process. Determine the different players and what are their values. Identify risks. Create lean canvas showing key points.
- Minimum Viable Product – their initial hurdle was can they connect sensors to a smart phone?
- Alpha – start with smaller test sites – they gave prototypes to building technicians. Measure everything. Then approach larger customers- once you have a sense of the value
- Beta – you’ve proven the values, now start the design process. Get feedback by showing design options to customers. At the end of this step, put it in fancy bag and ship it to customers.
- Iterate – rapid prototyping, react -> fix -> improve. – Their initial connector was not robust so they first superglued the leads so they would not disconnect. Then they got better leads. Improve the process. Go with small iterations.
- Scale – validate the design, then invest into the machinery to create a production line
Be ready to adapt and improvise: “always carry pliers and superglue”.
Luke Iseman @ycombinator.com then talked about startups from the point of view of a seeder. Y has funded 940 startups with107 in the last semi-annual batch, taking 7% of the company for $120 in seed capital. Most of their startups have been for software, but they have increasing seeded hardware startups.
After warning that “the only reason to do hardware is to change the world”, Luke spoke about the important steps in that will help you get funded as a startup and help the chances of long term success: Make; show; iterate; sell; grow; tell; fail
- Make– create a prototype (otherwise it’s just an idea), e.g. bodyport – scale to measure blood pressure within 5 seconds
- Show – get feedback. TeaBot – creates customs blends of tea. Find out what customers want
- Iterate – improve the product. Nebia created a better shower head funded through kickstarter. They went through numerous iterations to find something that looks great and uses 30% less water
- Sell – Transcend Lighting – high efficiency led light to grow crops indoors. Found a niche market of buyers for their high intensity/high efficiency grow lights.
- Grow – Click and Grow – indoor herb garden selling plug-ins to the basic product. They are expanding their market by creating new products, such as a kitchen work table with lower, lighted shelves to grow herbs.
- Tell – compelling story otherwise you become a commodity – Tully sells condoms, but with a story and a style.
- Fail – Luna sells smart mattresses covers. They were initially rejected by Y, but have grown.
The key is to sell the product!
Between the second and third presentations, Bjorn Bollendorff @Panono displayed his product: a sphere containing 36 cameras to take a full set of images surrounding the location. Each camera has a resolution of 3meg and images can be manipulated on an ipad. They are starting to ship limited editions units ($1499) and will next create a $599 consumer version.
The third speaker, Mark Yahiro @ Intel/RealSense spoke about the sensors developed by Intel to give PCs/tablets/phones human-like spatial knowledge to change the way individuals interact with computers. The 3d camera creates a point-cloud that maps the locations of solid surfaces/points in the vicinity.
Sensors can monitor heart rate and increase the difficulty of a game when one’s heart rate increases. An app by ItSeez3d does real time 3d scanning. An application being developed with BodyLabs sweeps each side of the body and retains your measurements to exactly fit clothing and for health/fitness monitoring. He showed a video of the sensors guiding drones through a forest while avoiding the trees.
Mark then showed how Google’s Project Tango integrates these sensors with others to allow one to walk through a world created in Minecraft and integrate the virtual and physical worlds in an AR shooter game.
Matt Turck interviewed Peter Weijmarshausen, CEO of Shapeways which receives files and 3d prints objects which they ship to the customer. They print 200k designs per month.
The company was founded in the Netherlands and originally outsourced all production. They moved to New York and in 2010 they started doing their own 3d printing. Currently, they print 50-60% of their products in-house.
Over the past seven years they have seen printing evolve. Initially, printing was done using FDM (stratus) printers. Then home printers arrived and demand for FDM evaporated. They have also seen the printing expand to include gold & silver (for jewelry), porcelain (coffee mugs), etc. They have also led the evolution of the business by using smart methods to squeeze greater efficiency out of their current printers.
Peter also talked about the growing market providing accessories for drones. He also talked about how Hasbro works with independent designers who create accessories for Hasbro toys and the profits are split amongst Hasbro, the producer and the designer. This approach takes advantage of the low cost of initial production allows the community to start testing products and quickly determine what resonates with the market.
He sees a bright future for 3d printing since there are still many inefficiencies that will be eliminates by new technologies and greater efficiencies of scale with the increased sales of printers. Currently, raw materials are still expensive. Machines are built at too low a volume. This is no full color plastic. Printing is too slow. He also sees improvements in scanners and the software behind scanners to eliminate much of the barrier to 3d printing in contrast to the complexity of current CAD software.
Additional observations were
- Conductive ink not available yet, but coming to 3d printing in a year
- Carbon3d will speed up the printing process
- Eventually 3d printers will be able to stack individual atoms (MIT research)
Night Café by Mac Cauley: lessons learned in creating a #VR experience
Posted on August 28th, 2015
08/27/2015 @Samsung, 130 Prince Street, NY
Mac Cauley talked about the inspiration and development of his award winning VR app, Night Café. Night Café is an interactive VR experience based on the paintings of Vincent #vanGogh and specifically his painting #NightCafé.
Mac originally was working on a live action film of a fictional painter inspired by van Gogh. The project evolved into a virtual reality application. He was also inspired by the works of Alexa Meade who paints subjects and photographs them.
Night Café (van Gogh, 1888) was used as his starting point for its expressionistic colors and absorbing perspective. Converting a single point of view into an immersive world required 3-d modeling of subjects and items along with a reimagining of the corner that is not visible in the painting. For these Mac studied the fixture of that period and modeled the individual subjects in three dimensions.
He adhered to certain design rules. These included making each object unique and emphasizing the textures which is one of the distinguishing characteristics of van Gogh’s paintings.
He used many tools to the create objects, object skins, animation, etc.
Some of his main tools were: Maya, Unity, Mudbox, RapidRig.
He also talked about the challenges and dead ends as he developed the application
When animating, he initially used Kinect mocap which he considere cool, but felt that the quality was inadequate. He used keyframing and found the process slow but worth it.
He tried shading using particles, but found they did give sufficient detail and ran too slowly. He ultimately used flat shading with no lighting since he could take colors from the painting and it provided good performance.
He initially designed a complex set of controls to move through the space, but eventually realized that simple controls (such as tap and hold to move forward) using a touch pad worked best.
The VR experience was enhanced by optimizing-texture aliasing, mesh batching, texture/audio compression, reduce particle counts, etc.
He summarized his main lessons learned:
- characters are very interesting to see in 3d
- particles are awesome
- movement is tricky – slow down movement, simplify controls, eliminate acceleration (stay or move only)
- Note 4 is powerful enough
- Stylized worlds can still be immersive.