This summer, the HPE AI at Scale Community Team went to several conferences, including MLSys, CVPR, and ICML. We saw exhibitions, attended talks, and got caught up on important papers. Here are the highlights.

Robotics

At CVPR, there were a ton of robots on the show floor, including Unitree, Boston Dynamics, and this cute little Matic robot doing SLAM (simultaneous localization and mapping) in the conference venue:

At ICML, one of the featured speakers was Chelsea Finn, Associate Professor at Stanford. She gave a talk about how hierarchical policies are used in robotics to help them learn to do complex tasks, from filling a plastic bag with objects to performing robotic surgery. Usually, it’s hard to get end-to-end data for the entire task, so Chelsea’s lab created a framework that allows updating only one model out of two.

Autonomous Driving

Zoox, an autonomous taxi, and Gatik, autonomous trucking, were both at CVPR!

3D reconstruction and Digital Twins

Meta, Captions, and TikTok were the highlights here.

Tools of the trade

And of course, many solutions for GenAI & Data Pipelining across the board, including our very own Determined AI & Pachyderm.

Research trends at CVPR & ICML

The papers at CVPR and ICML covered a wide range of topics at the cutting edge of machine learning research. While CVPR generally has a greater focus on computer vision, the two conferences had many common themes. Below, we highlight a few of those themes, and a few interesting papers from each category.

Large Language Models (LLMs) and Vision

• VideoLLM-online: Online Video Large Language Model for Streaming Video

• Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition

• Koala: Key Frame-Conditioned Long Video-LLM

• VideoPoet: A Large Language Model for Zero-Shot Video Generation

Diffusion Models

• Diffusion Models Encode the Intrinsic Dimension of Data Manifolds

• Fixed Point Diffusion Models

• Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models

• Align Your Steps: Optimizing Sampling Schedules in Diffusion Models

Graph Neural Networks

• Learning Structure-from-Motion with Graph Attention Networks

• Cooperative Graph Neural Networks

Federated Learning & Privacy

• Traceable Federated Continual Learning

• Privacy Attacks in Decentralized Learning

Applications of ML

• VAREN: Very Accurate and Realistic Equine Network

• MatSynth: A Modern PBR Materials Dataset

• Transcriptomics-guided Slide Representation Learning in Computational Pathology
• CoralSCOP: Segment any COral Image on this Planet
• PolarRec: Improving Radio Interferometric Data Reconstruction Using Polar Coordinates
• Estimating Canopy Height at Scale
• GeoAB: Towards Realistic Antibody Design and Reliable Affinity Maturation
• OxyGenerator: Reconstructing Global Ocean Deoxygenation Over a Century with Deep Learning
• CHEMREASONER: Heuristic Search over a Large Language Model’s Knowledge Space using Quantum-Chemical Feedback
• Teeth-SEG: An Efficient Instance Segmentation Framework for Orthodontic Treatment based on Multi-Scale Aggregation and Anthropic Prior Knowledge

Efficient AI

• Cache Me if You Can: Accelerating Diffusion Models through Block Caching
• DeepCache: Accelerating Diffusion Models for Free
• Accelerating Neural Field Training via Soft Mining
• PikeLPN: Mitigating Overlooked Inefficiencies of Low-Precision Neural Networks
• GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
• KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache
• Extreme Compression of Large Language Models via Additive Quantization
• Communication-Efficient Federated Learning with Accelerated Client Gradient

Upcoming events

We’ll be at PyTorch Conference 2024 in San Francisco! Check us out.

We’ll also be hosting our very first MLOps Community meetup in late October! Stay tuned for more details.

Stay up to date by joining our Slack Community!