Nvidia GTC — ML/AI/Robotics
I attended the Nvidia AI conference in person back in 2016 and really enjoyed it — in particular trying out numerous VR experiences. This year you can attend for free online.
I’m sharing highlights from some of the talks that I watched.
Keynote
The keynote is presented by the Nvidia founder and CEO Jensen Huang. It contained lots of interesting examples of how to use their Nvidia hardware and software to build many real world applications: detecting oil spills, forest fires, navigating ice flows, and detection fall risks in hospitals.
They did a great job showing their vision in interesting ways. Jensen gave the presentation from a virtual world built with Nvidia’s Omniverse tools. Not only did his virtual background dynamically rebuild itself as a segue between topics, but in The Making of the GTC Keynote video they shared that every keynote slide was built in Omniverse — it looked like some were 3D models, projected into 2D slides.
Intelligent Assistants
In the keynote they demoed “Toy Me”, which is a digital avatar of Jensen that showcased their intelligent assistant, coupled with a photorealistic avatar that responds real time, with face animation to accompany the audio. The speech still sounded a bit wooden, and there were slight pauses before a response begun, but overall the effect was impressive.
They also showed a video demo of “Tokkio”, an intelligent assistant talking kiosk for a restaurant.
This is my first time seeing an intelligent assistant being used this way, and it seems very likely that we’ll see widespread commercial adoption of these within the next 10 years.
Physics ML
Apparently physics-enabled machine learning (ML) is going to unlock some huge capabilities in the near future. Climate modeling was covered a few times as an example.
Climate simulation is much harder than weather simulation. Weather simulation models atmospheric physics, and its predictions can be checked regularly — climate change predictions spans decades, which makes that model training feedback loop more difficult. In addition, the scale of climate simulations are currently beyond our reach.
In another physics ML example, they showed a cool video animation simulating a chemical reaction.
These capabilities should apparently help speed up discovery of new drugs.
Robotics
There was an interesting comparison between robotics and edge computing. They have very similar functions: they both typically require sensors, offline/on-device computation, and can often use the same technology pipelines.
Medical instruments are increasingly using robotics.
Nvidia are launching their Clara Holoscan AI Platform for medical devices. This is one of many situations where Nvidia are selling a combination of both hardware and software services on top of them.
Nvidia have an Isaac SDK:
Build and deploy commercial-grade, AI-powered robots. The NVIDIA Isaac SDK™ is a toolkit that includes building blocks and tools that accelerate robot developments that require the increased perception and navigation features enabled by AI.
As you’d expect, at the base of this platform is the Nvidia hardware. :-)
In the talk “Accelerating Robotics with NVIDIA Isaac ROS”, they talked about the history of ROS/ROS2
and they explained many of the various features, like image rectification libraries (it wasn’t clear to me if any of these were unique to the Isaac SDK, or whether other open source/ROS code exists for them already).
If you want a cool little robot to experiment with, you can 3D print a Nanosaur:
🦕 nanosaur is the smallest NVIDIA Jetson dinosaur robot, fully 3D printable.
In 10 x 12 x 6cm and 500g use only a power-bank and works in ROS2 is simple to build and simple to use.
Isaac SDK is intended to be an open platform that you can build on. As you can see from the slide earlier above, Isaac has a simulation layer, which feels similar to what the Omniverse tooling provides.
Goal is for the robot not to know it is in the simulation.
Autonomous Vehicles
Every AI conference needs to mention autonomous vehicles. :-)
I love seeing videos of self driving cars, especially with their sensor inputs and data visualizations.
DRIVE Hyperion is another of Nvidia’s platform plays, combining hardware and software:
NVIDIA DRIVE Hyperion™ is an end-to-end development platform and reference architecture for designing autonomous vehicles (AV). It includes multiple DRIVE AGX Orin™ Developer Kits, each powered by an Orin SoC delivering up to 254 TOPS, a variety of sensor modalities to enable 360-degree surround view, and extra compute power to offload data recording and replay. The system is verified, calibrated, and synchronized out of the box to support rapid scaling from development to production.
They also showed how intelligent assistants can be used as either a concierge or chauffeur in the vehicle.
It’s very interesting to see how intelligent assistants could be used in such a wide variety of products. I went to a Google talk a few years ago which talked about voice becoming the universal user interface, since it works everywhere. We aren’t yet at the point where intelligent assistants can truly hold a conversation, but they do work really well for narrow applications.
