PyTorch Mobile, Pixel 4, edge ML benchmarks, the state of ML frameworks in 2019, building richer apps with ML Kit, and more


PyTorch 1.3 adds mobile, privacy, quantization, and named tensors

A huge announcement from the PyTorch team with regards to machine learning on mobile. The release of 1.3 includes experimental support for a number of mobile-focused features: a deployment pipeline, model quantization, and several front end improvements (like named tensors) that will lead to cleaner code. The experimental build of the end-to-end deployment flow works with Python, with support for both iOS and Android. There will certainly be some kinks to work out in the early days, but it’s highly encouraging that PyTorch is actively working on a solution to help developers deploy their models to mobile. [Read More]

Early takeaways and experimentation with PyTorch Mobile

Julien Chaumond, cofounder and CTO of Hugging Face, which has been a recent leader in all things NLP, shared some early impressions with the experimental PyTorch build, establishing some early benchmarks and comparisons on iOS. Julien’s recent Twitter thread recaps these findings. [Read More]

Pixel 4 announced: Face Unlock, Motion Sense, Pixel Neural Core, and more

Google announced their newest line of Pixel phones at their launch event Tuesday, unveiling a slew of new features and capabilities (many of them leveraging on-device machine learning). The Pixel 4 now has Face Unlock (similar to Apple’s Face ID) and Motion Sense, which creates a 2 ft-diameter hemisphere of spatial awareness around the phone. The Neural Core is an improvement on the Pixel 3’s Visual Core, signifying that Google is moving beyond on-device computer vision (i.e. photo enhancement in the Pixel 3) towards speech and NLP tasks. Lots more detail in this hands-on look from The Verge. [Read More]

The state of machine learning frameworks in 2019

Given the big news from the folks at PyTorch, this breakdown of the PyTorch vs TensorFlow debate from The Gradient is quite timely. From the author: “My analysis suggests that researchers are abandoning TensorFlow and flocking to PyTorch in droves. Meanwhile, in industry, TensorFlow is currently the platform of choice, but that may not be true for long.” Lots of interesting discussion that attempts to back up these claims. [Read More]

Powerful computer vision algorithms are now small enough to run on your phone

Essentially, this has been true for quite a while, but this article in the MIT Tech Review focuses specifically on advances made in processing video on-device, which is generally more resource-intensive than working with still images. The new method, which can train models up to 3x faster than the current state-of-the-art, could significantly reduce latency and computation costs across a wide range of on-device use cases. [Read More]

Machine learning edge devices: Benchmark report

Late last week, Tryolabs released and extensive benchmark report for ML on edge devices. Their team did a great job of prefacing the data with a discussion the what and why of edge ML, arguing that we’re just now entering “an era of edge computing and edge devices”. They benchmarked 5 particular edge devices via image classification: the Jetson Nano, Coral Dev Board, Neural Compute Stick, Raspberry Pi, and the 2080ti NVIDIA GPU. [Read More]

Machine learning development on Apple platforms

Nick Arner provides an excellent overview of the current state of ML on Apple platforms, ranging from Core ML and Create ML to model conversion tools and resources to help developers start building. If you’re looking for a high-level review of machine learning on iOS, this will be a great resource to get started. [Read More]


[GitHub] pytorch / ios-demo-app

PyTorch iOS examples, updated with the release of PyTorch Mobile. [Explore]

[GitHub] pytorch / android-demo-app

PyTorch Android examples, updated with the release of PyTorch Mobile. [Explore]

[*Fritz] fritzlabs / fritz-examples

Our new examples repo — a consolidated collection of experiences utilizing machine learning models from Fritz AI. [Explore]


[Video] Building richer app experiences with machine learning

From the ML Kit team at this year’s Firebase Summit, this video covers ML Kit’s ready-to-use APIs, custom model creation with Auto ML, and model deployment. [Learn More]

[Heartbeat Collections] Machine Learning on iOS: Model management and optimization

A catalogue of Heartbeat posts covering techniques for managing and optimizing machine learning models on iOS. [Learn More]

[Google Colab] TensorFlow 2.0 + Keras Crash Course

An official introduction, crash course, and quick API reference for TensorFlow 2.0. [Learn More]

Create ML for iOS — Increasing model accuracy

Navdeep Singh takes us through several techniques for improving model accuracy metrics when building iOS-ready ML models with Apple’s Create ML. [Learn More]

Editor’s Note: Heartbeat is a contributor-driven online publication and community dedicated to exploring the emerging intersection of mobile app development and machine learning. We’re committed to supporting and inspiring developers and engineers from all walks of life.

Editorially independent, Heartbeat is sponsored and published by Fritz AI, the machine learning platform that helps developers teach devices to see, hear, sense, and think. We pay our contributors, and we don’t sell ads.

If you’d like to contribute, head on over to our call for contributors. You can also sign up to receive our weekly newsletters (Deep Learning Weekly and Heartbeat), join us on Slack, and follow Fritz AI on Twitter for all the latest in mobile machine learning.