In our latest blog post, we discuss some of the theoretical and practical considerations that deep learning engineers run into as they attempt to scale training beyond a single machine. There’s a lot to get right to even get functional distributed training off the ground. Once you do, there is a rich space of optimizations to navigate whose efficacy can depend on everything from your model architecture to your network topology.

By the way, here’s how easy it is to enable optimized distributed training in Determined AI:

Recommended reading:

• Our Chief Scientist Ameet Talwalkar weighs in on “Green AI”, and the imminent need to shift towards energy-efficient model training on IEEE Spectrum.

• Harvard Business Review discusses ways that companies should be thinking about navigating the landscape of AI Platforms. In it, our CEO Evan Sparks advises against a duct-tape oriented approach.

• Two new pieces of work out of Ameet’s lab at CMU focused on explainable AI. The first focuses on a new approach to learn ML models, e.g., deep learning models, in such a way that they are more amenable to subsequently be explained by black-box explainers like LIME. This approach is validated in part by a cool user study. The second work looks at explainability in unsupervised learning settings, motivated by common clustering workflows such as those performed for single-cell RNA data.

• This paper presents our novel massively approach that powers our product’s hyperparameter optimization and NAS functionality.