CodeDrivenNYC: caching web pages, #NLP, bringing #coding to the masses
Posted on November 20th, 2015
11/19/2015 @FirstMark, 100 Fifth Ave, NY
The first of the three presenters, David Mauro @Buzzfeed spoke about creating Mattress, their first open source IoS framework. Mattress caches web pages for later, off-line consumption. It also makes it appear that the page is loading quicker when online.
David spoke about the hurdles implementing this product
- How do we download an entire web page?
- How do we provide the content back to user
Their first decision was to download the URL using UIWebView and then capture all requests as they come through using NSURLProtocol. UIWebView runs on main thread and is resource intensive, but the alternative to manually parse the HTML and the JS. They download the URL using UIWebView and then capture all requests as they come through using NSURLProtocol. But WKWebView does not handle NSUIRLProtocol and there is a bug so you cannot just save another NSURLCache. They use commonCrypto to retain the URL, with the name hashed so even the longest name is uniquely identified.
They also need to know when a page if done downloaded. Automated solutions have tendencies to either terminate prematurely or not terminate at all. Instead, they ask the user when the download is done.
How to provide the content back to NSURLProtocol? First ask the user if they are offline. If so, they retrieve the page from the custom offline cache. If they are online, the system reloads the initial request.
The system was designed as a simple API that can be run either in foreground or in background fetch. The background fetch needs to be monitored carefully so it does not use too much of the battery or slow the processing excessively.
The second speaker, Rob Spectre @Twilio demonstrated how easily applications can be made interactive using the Natural Language Processing tool, Textblob running in python.
Rob showed how to create an app that receives SMS text messages and changes its response based on your message. In just a few lines of code, Rob showed how the response can be differentiated based on the length of the message, it’s sentiment, it’s sentence structure, etc.
Ryan Bubinski @Codecademy asked the question “What is code?”
As an overview of the many ways to answer that question he recommended the 38,000 word article written by Paul Ford in Bloomberg June 2015
He summarized his view by saying that code is a lever that is becoming more powerful every day. As an example, he mentioned OpenFace, an open source program which uses a neural net for face recognition.
Making this lever available to more people requires either
- Making coding easier or
- Making it easier to learn how to code