PyTorch Developer Conference 2018 - PTDC - part 1

This is my cliff notes style blog about my experience at PyTorch Developer Conference that happened on October 2, 2018. This is part 1 because as I was going through my notes, there was so much content and things to read and explain more, that it couldn’t easily fit into one blog.

Talks

The talks section is in loose chronological order of the speaker schedule

Here are the links to all videos from the conference:

• PyTorch Developer Conference Part 1
• PyTorch Developer Conference Part 2
• PyTorch Developer Conference Part 3

Keynote Jerome Presenti

Recent breakthroughs in AI are things such as DensePose. This library has been recently open sourced by FaceBook.

Today FB is announcing PyTorch 1.0

Current flow to get PyTorch into production is:

Pytorch -> ONNX -> Caffe2

PyTorch 1.0 is about making this a more seemless process with the torch.jit module.

We need testing and tooling for ML, like software development 20 years ago. A seamless process to get PyTorch into production should exist, so torch.jit was created.

Hardware breakthroughs like the volta have accelerated ML research.

Operator fusion now speeds up training times.

Deep Dive on PyTorch 1.0

The goal of PyTorch 1.0 is to make putting PyTorch models into production as seamless as possible. For this, the PyTorch team is also considering:

• Research and flexibility is central to PyTorch.

• Production constraints are:

  • hardware efficiency
  • scalability
  • cross platform

Currently ONNX is the way to put PyTorch models into production.

With PyTorch 1.0 comes the introduction of the torch.jit module. This module allows the developer to write code once, and with the use of torch.jit.trace decorators and other helpers from the module, the Python PyTorch model will be output to run independently from Python, for instance in a C++ program, and be production ready.

PyTorch model development workflow that’s possible as of 1.0:

• experiment with models in Python
• extract the model using a torch.jit.script
• optimize and deploy

Torch scripts do operator fusion for algebraic simplification.

torch.jit module

Functions are decorated with torch.jit.script, while subclasses of torch.jit.ScriptModule use a torch.jit.script_method decorator. Also:

• records control flow
• uses a subset of Python
• more info here: https://pytorch.org/docs/master/jit.html#torch.jit.ScriptModule

torch.jit.trace does the following:

• runs code straight through
• records what happens
• can run later without Python present
• more info here: https://pytorch.org/docs/master/jit.html#torch.jit.trace

Models can then be saved to disk.

Tutorial for loading a PyTorch module to C++

PyTorch library hierarchy described as:

Aten
 | 
Autograd
 |      \
 |    Python script
C++ extensions

Custom C++ exetensions for PyTorch can be written

• Uses PyBind
• setup.py can load the custom extension

Deep learning C++ bare metal for ML

All links available at pytorch.org/cppdocs

Distributed training uses C10D

• performance driven C++ backend with Python frontend
• uses futures with async_op=True
• distributed_c10d.py
• communication overlap
• smart gradient sharing

torch.distributed is deprecated in favor of C10D

Fairseq

Fairseq provides a sequence modeling toolkit

Seq2Seq library that has been instrumental at FB

Won the Global Machine Translation 2018 competition

Trains machine translations (MT) in both directions

PyTorch Translate

https://github.com/pytorch/translate

• predict content language
• predict view language

Uses this flow

Alten -> decode
            \
            C++ Beam search
           /
Encoder -

FBGEMM for quantization

PyTorch Text has DataLoaders and abstractions for NLP

AI at Facebook

2.2B Users

300+ trillion predictions made per day

FB AI Infrastructure runs many experiments per day

Things that FB needs to achieve it:

• AI as a service
• Data platform
• AI foundation

Current library relationship

PyTorch - research
 |
ONNX - transferring
 |
Caffe2 - production

Things that help scaling ML learning

• increased batch sizes
• increased learning rate
• overlap with training and balance

More notes about the current ML environment

Models are needed for a highly fragmented hardware world. Mobile is highly fragmented and faces size constraints. Python isn’t always present, so production models should run without needing Python.

Quantization has been proven to speed up training without hurting model accuracy.

LibTorch C++ frontend for PyTorch is now available. Or, for download here https://download.pytorch.org/libtorch/nightly/cpu/libtorch-macos-latest.zip

People I met

The community was so friendly and welcoming. Coming from a software development background and not data science, with my knowledge of data science from online classes I was able to communicate, learn and understand everyone at the conference.

Some people I met were:

• Conor - ML Intern
• Tom - CTO at Paperspace
• Francisco - ML Intern at fast.ai
• Fedro - Data scientist working with Yann LeCun
• James - core PyTorch Dev

Next

I’ll be working on part 2 of this blog in the same style as this one. It will be a continuation of material presented in the conference with lots of links. More great PyTorch to come :)