#TensorFlow and Cloud Machine Learning
Posted on June 7th, 2016
06/06/2016 @Audible Inc, 1 Washington Place, Newark, NJ 15th floor
Joshua Gordon @Google talked about #MachineLearning and the TensorFlow package. TensorFlow is an open source library of machine learning programs. Using the library you can manipulate tensors by defining graphs that are functions operating on the multivariate structures.
The library runs on Linux and OSx. The library runs on Windows using Docker. Support for Android is on the way. Joshua showed several applications including one that repaints an image in van Gogh’s style by merging levels from the network identifying colors from the original image with layers from a second network trained on the painter’s style.
Next, Yufeng Guo @Google talked about out-of-the-box machine learning APIs to classify images. Google has a cloud vision API and will shortly release a speech API.
The vision API imports a jpg file and outputs a description in JSON format including items identified and the confidence that the items are correctly identified. It also gives the coordinates of items identified and links to the full description of the items in Google’s database. The face detection routine also outputs information such as the rollAngle, JoyLikelihood, etc. The service is free for up to 1000 requests per month.
Review of Google I/O 2015
Posted on June 26th, 2015
06/24/2015 @Google, Chelsea Market, 9th & 15th St, NY
Speakers presented their impressions of the #Google I/O meeting that was held in San Francisco, May 28 to 29.
First, Nitya Narasimhan gave a rapid overview of the directions of Google’s research over the past year. These initiatives include
- Advanced Tech and Products (ATAP)
- Chrome browser update
- Polymer & web components
- Internet of Things
- ATAP – advanced tech & products included
- Jacquard – Use fabric as a touch sensor
- Ara – modular smart phone- with hot swapable components
- Soli – use radar for gestural interactions – interactive interfaces can be on anything
- Abacus- replace passwords with a series of gestures and your unique actions
- Vault- security archive
- Tango – indoor location sensing
- Chrome browser update – focus on R.A.I.L. performance standards (click here for a full presentation or Paul Irish’s keynote) which are performance goals to improve the UI. Other things to improve the user experience
- devtool – Filmstrip shows you what the page looks every second as the page loads.
- web security using a “speed dating” analogy as you move from one web app to the next.
- Polymer & modern web APIs. (web components) :
- you can create custom HTML tags
- you can export HTML components to other HTML files
- templates to create Polymer elements
- shady DOM – faster than shadow DOM but less capable – for older browsers
Polymer has families of custom components that make it easier to create custom HTML tags
- Iron- web components
- Material design
- Go web – Google services
- Gold targets ecommerce
Polymer starter kit is available with app templates
Material design is now available for both web and mobile.
- Cardboard VR
- Cardboard 2.0 now available for iPhone.
- Can handle bigger phones and the device is easier to assemble
- Expeditions is an initiative to create content for cardboard for education.
- Jump – a goPro hardware setup which captures a 360 degree view with software to stitch it together. Output to cardboard and Youtube.
- SDK for cardboard- example includes a walkthrough for building with Unity.
- Design- designing for virtual reality – see the online demo – see cardboard design lab for guidelines
- Tango – 3d motion tracking and depth scanning
5.Internet of Things
- Nest – thermostat
- Thread group – industry consortium looking to standardize a web and communication links between objects
- Brillo – smallest version of Android OS to run on almost anything – a tool for building with Brillo is expected out soon
- Weave is the messaging system
- Speech and neural networks
Next, Dario Laverde covered Android M, the upcoming version of the OS. It’s also called MNC, short for ‘Macadamia Nut Cookie’ and has an expected release date in the 3rd quarter
- Google Now can be done from any app. This means one can ask for the lead singer of a song as an app is playing that song
- App permissions : at run time you are prompted if you want to access camera, etc. – similar to that on the iPhone. Developers need to check their apps on the emulator to see if this affects older apps. Can also view permissions by app or by capability (e.g. camera).
- Voice Interactor allows for confirmation of actions by voice
- Fingerprints for authentication
- Android backup – backups up all data by default
- Google Play Services 7.5 – one can build deep links to features within your app.
- Can Cast the screen image to a remote display;
- Smart lock so passwords entered on one device do not required repeated password entries across web and mobile devices
- new exercise types added to types in Google Fit database
- untouched devices causes apps to become “inactive”. However, developers can whitelist an app so it does not go to sleep;
- Android design support library: notification icon can now be a resource id or a bitmap (not just jpg and png files)
- new version of Android Studio including real time step-by-step debugging in C++ in Android NDK
- Styluses are now supported on tablets
- Tools: Systrace is a tool to locate problems. Also new compiler optimization
- External storage such as USB device supported by adb
- Graphics – separate TORCH light from the camera controls
- Audio – midi interface
Other developments related to Android include
- New Android Developer Guide
- Android wear now allows maps to be displayed on a watch face see Github/googlemaps…
- Project Tango
- Tango tablet contains the sensors for motion tracking, area learning and depth perception.
- Google announced 3 Contests to build Tango-powered apps in utility, VR or entertainment
- Tablet sells for $512
The third speaker, Ralph Yoozo spoke about Firebase, an online, real time database. Firebase is easy to setup since it requires no server side code to set up security. It can receive data using web sockets or virtual infinite web pages. He has built two applications using Firebase
- A web page to show runners their times (adjusted for their start time) as they crossed the finish line
- The bills in discussion in the NY state senate: see https://nysenate.firebaseapp.com/# s
Ralph also noted
- Firebase is open so everyone can read and write to it, but this can be adjusted
- Can run a curl command form the command line to test the app
- Have a simulator page to debug the code
Ralph also briefly talked about Universal Second Factor which promises to offer better security than just a password. It is a small device (can attach to key chain) that provides a second layer of security in addition to your password. It uses a FIDO protocol.
The meeting was concluded by two brief talks
In the first, Howard Goldstein@NYTimes talked about Smart Lock, which integrates Chrome’s password manager so it extends to Android. Howard said it was very easy for the New York Times to integrate Smart Lock into their applications
- Needs Google APIClient
- Request credentials (the password)
- If succeed, can auto-login
- If fail, some credentials may not have passwords
- If fail, might have multiple accounts on the device – have the user select an account
- Can push credentials to Google so user does not need to enter them again
The second brief talk was by Anna Yan, a first timer’s visit to I/O. She spoke about the two devices demonstrated:
- Exiii makes robotic arms costing $200 They capture motions to an android phone and can be customizable using 3d printed
- Neosensory maps sound patterns to different locations on a vest making use of the sense of touch. The vest can be used as a sensory substitution for the deaf or in extremely noisy environments.
Mar GDG Meetup: #ServiceWorkers, Advanced #TextViews and ng-conf Recap
Posted on March 19th, 2015
03/18/2015 @Google, Chelsea Market, NY, 75 Ninth Ave, NY
Presentations on #TextView and #ServiceWorkers were supplemented by four shorter presentations.
Chiu-Ki Chen spoke about how Android TextView in the UI can be used to create a wide variety of exciting visual effects.
She first concentrated on how the contents of a TextView can create text shadows, glowing text, custom fonts, color gradients fonts and text pattern coloring
She next showed how spannable strings can be used to create strings of characters with variety of adjustments in font height, color, clickability etc. within a single TextView. A single TextView can therefore create the look and feel of tagged segments in an Html document.
As a final touch, Chiu-Ki showed how to create text on a lined page by combining text and graphics on a canvas. She also showed how emoji unicodes can be remapped to images which can be resized and positioned like characters.
- Responds to events
- Applies to all pages under a scope and persists when tabs are closed
- A cache is exposed to the service worker, so pages can be used off-line.
- Can modify CSS.
- For security reasons, it only runs on https locations
Programming. Service workers are based on Promises (a syntax which is an alternative to callbacks for asynchronous code.) In the code, an asynchronous process is called and returns immediately with a promise that will eventually be fulfilled. Once the promise is fulfilled, the code starts the next process. Debugging can be done in Chrome using the web page: chrome://serviceworker-internals
if (‘serviceWorker’ in navigator) …
so the Service Worker only runs when the browser can handle it.
Service worker should live at the root of your domain since it only has control over network requests that are within that part of the tree.
Things you can do. If you are offline and the video you requested is not available, it will return the previous video you downloaded. It can write a custom off-line page. It can trigger a push notification so long as Chrome is running, even if the page is not active. It can send notifications to a wearable watch.
In the shorter presentations
- Alex Qin – @CoalitionForQueens – www.c4q.nyc talked about their program to teach adults to code. They are looking for instructors.
- Elcin Erkin – updated the audience on the latest developments in AngularJS from the ng-conf 2015 in Salt Lake City Utah. The main discussion was about migrating to Angular 2.0. The upgrade will feature
- Better performance and design.
- New TypeScript created jointly by Microsoft and Angular
- Speed up using unidirectional bindings.
- Changes in syntax and semantics.
- A new router to better serve complex applications.
- The Angular team has taken over the Material Design component.
- Incorporate web components.
- Ray Tsang – In honor of pi day (3/14) he calculated pi to 100B digits in 5 ½ hours on a multicore processor.(I wonder what the distribution of digits looks like)
- Dario Laverde– showed how easy it is to create Wearable watch faces using Android Studio. In less than a minute he created a face without writing a single line of code.
GDG web development topics including #AngularJS & #MaterialDesign, #Hypergrid & #Polymer
Posted on February 22nd, 2015
02/20/2015 @ Google Chelsea Market Office, 75 9th Ave. NY
There were two technical presentations. The first by Steve Wirts@OpenFin on “HyperGrid: An Open Source HTML5 Grid Component”. The second by Daniel Zen@ZenDigital on Material Design In Practice With AngularJS“.
There were also two shorter talks. The first, “Get Involved: CS Education in NYC” Speaker: Cindy Gao (Chief of Staff, CSNYC) and the second by Jeff Posnick @Google on “sw-precache: A Service Workers module for offline website operation”
In the first talk, Steve Wirts demonstrated the advantages of using HyperGrid: one can scroll, search and zoom rows and columns in a spreadsheet with over 1 million rows.
Hypergrid is an open source, ultra-high-performance Chrome-based browser designed to display very large grids such as the portfolio holdings of large banks. It makes extensive use of polymer, an open source tool to create and manipulate web components. Using polymer, Hypergrid encapsulates the tags connected to the various links needed to speed performance while simplifying the code. Web components make it easier to label the tools within a page.
Hypergrid also uses Vulcanize to flatten dependencies and reduce network requests.
Steve next talked about lessons when creating HyperGrid. These include:
- Knowing good game design helps in all development – He recommends the book “Html5 Game Design Insights”
- Be sure you understand the ins and outs of device pixel density
- Talking to graphics context is expensive so only iterate/render what an item is visible
- Border rendering, background fill per cell is extremely expensive, so employ polymer so updates are done on rows or columns, not on individual pixels
- Test your changes on all browsers
- Vulcanize is NOT an intermediary tool. Use it for final deployment only
- Use a rectangle based selection model, not an irregular shape
- Take advantage of Polymer plugin code
Daniel Zen talked about Material Design, an increasingly popular paradigm for UI design. Material Design gets its name and inspiration from the physical world inhabited by materials that we manipulate and whose physical appearance clues us into its function.
More specifically, Material Design is a system for visual, motion, interaction design based on the following notions:
- material is the metaphor
- bold, graphic, intentional images – core functionality should be immediately apparent
- motion provides meaning especially when the user initiates the motion. Animation highlights the process of the change as the interface moves from one mode (e.g. button pressed to open a window) to another mode (e.g. the new window appears)
To implement these notions, flat sheets appear in a 3-dimensional environment with sheets placed on a flat surface. This effect is enhanced by using keylights, ambient light and shadows to indicate distance to a base level. The movement of each sheet signals to the user concepts of light/heavy, flexible/rigid, small/large. Popups appear to grow from the button initiating the popup. Bold images are enhanced by limiting the colors to two pallets: a primary (3 colors) and a secondary (1 color). Other visual effects such as a scrim behind the text make the text easier to read.
Daniel next showed how Material Design interfaces can be implemented using two different methods.
One uses AngularJS material directives which create tags starting with “md_” and take advantage of css layouts built on flexbox.
An alternative is to use web components implemented using polymer (loaded first) and angular.
Another alternative is Materialize which is a css library for material design which does not need angular or web components.
Cindy Gao spoke about the opportunities to help teach programming part time in the public schools.
Jeff Posnick spoke briefly about Service Workers, which can be attached to Chrome to make web pages look more like native applications when displayed on a mobile device. The service worker is attached to the browser once the page is loaded and will stay there when the page is brought back much later. This gives additional capabilities to a web site including detecting when an internet connection is not available and executing fallback actions, such as getting information from a local cache (which can be set up at build time).
The Service worker can also schedule a background sync and acts like services that run independently on your chrome browser. Jeff will be providing more details in future meetings.
In a previous post I noted that Microsoft also has an initiative to make web-based mobile apps look and feel more like native apps: Web App Templates (WAT) which provides tools for mobile developers to locally modify the look of pre-existing web pages.
Solve for X: proposals to solve big problems #SolveForXnj
Posted on January 7th, 2015
01/07/2014 @ Google 111 8th Ave, NY
Seven speakers a spoke about their ideas to create a world-changing product. Four were medically related. The others covered education, green technology and virtual travel.
Leonard Flom – kill ebola in vivo – currently quickest way to kill ebola is using ultraviolet-C. Your own immune system kills you. (“cytokine storm”).
He proposes using kidney dialysis machinery to exposure blood to UV-C
- Kill Ebola on surfaces in air
- Harmless to humans
- Stimulates production of immune bodies
Challenges – can dialysis machines be successfully deployed out in the field in a country with primitive infrastructure such as Liberia? The method only kills ebola in the blood stream, not those already in tissues.
Bodo Hoenen – how to teach the forgotten children? Distribute autonomous tablet-based apps. Design a platform that keeps a profile and searches for educational resources and a social network. Not a technology problem. Open architecture so anyone can contribute.
Challenges – who will define what is presented. How does on overcome the political resistance of local rulers that take advantage of ignorance.
Wenting Zhang – personalized medicine to treat cancer (cancer is not a single disease). One disease model is multiple myeloma (plasma cancer). Currently no cure. Disease cannot be cured within the body. Must maintain cells outside the body. Use a mouse model – inject cells from patient bone cells into a mouse, but this often fails. Idea is to use external human tissue microenvironment to give a snapshot the trial drug tests.
Challenges – need to replicate the in vivo environment externally to determine if the response to drugs is the same externally and internally. Can this model be applied to other types of cancers.
Richard Riman – Everyone can apply technology that is sustainable. Problem is that countries that are modernizing are using outdated technology. We need sustainable, portable technology. Concrete (CO2 production) and trash are the building blocks of this toolbox. 1. Use carbonate cement concrete which is dug from the ground and hardens when exposed to CO2. 2. Convert trash into CO2 and water.
Challenges – availability of ingredients, durability of machinery, other (potentially toxic) outputs of the trash converter.
John Kulp – Molecular engineering for the war on drug-resistant microbes. Microbe proteins change over time so drugs don’t bind. Ligand engineering enables the design of drug molecules to meet precise requirements. Define requirements and simulate fragment maps to find drugs that will bind. But the process is manual and slow. They want to scale up the ligand engineering using cloud computing.
Challenge – this is an engineering problem, but what is the business model? How does one test that the best working ligands do not produce toxic outcomes for the cells in the body?
Vincent Ogutu – simulated teleporting.
Main problem is bandwidth to transmit holograms across internet. Better to transmit only essential information – create an avatar. Then merely need to send the skeleton.
Challenge – how do we interact with a hologram? Do we need new social conventions?
Norbert Guzman – portable instrument for identify early warning biomarkers of a disease.
He wants to build a device that can take in blood, cells, urine, etc. Could provide history of many biomarkers over time and could be placed in an ambulance for very early diagnosis.
Goal is 20 minutes to get results and devices eventually in every home.
Challenge – cost.