Skip to main content

Chapter 3: Isaac Sim Architecture

Learning Objectives​

By the end of this section, you will be able to:

  • Understand NVIDIA Isaac Sim architecture and Omniverse foundation
  • Compare Isaac Sim with Gazebo for robotics simulation
  • Configure Isaac Sim for humanoid robot development
  • Leverage GPU acceleration for faster-than-real-time simulation
  • Understand photorealistic rendering and sensor simulation

Introduction​

NVIDIA Isaac Sim is a GPU-accelerated robotics simulator built on NVIDIA Omniverse. Unlike CPU-based simulators like Gazebo, Isaac Sim leverages RTX GPUs for:

  • Photorealistic rendering (ray tracing, global illumination)
  • Massively parallel physics (thousands of robots simultaneously)
  • Synthetic data generation (for AI training)
  • Real-time sensor simulation (cameras, LiDAR with realistic noise)

This section introduces Isaac Sim's architecture and its role in Physical AI development.

Isaac Sim vs. Gazebo​

FeatureGazebo (Fortress)Isaac Sim
Physics EngineDART/Bullet (CPU)PhysX 5 (GPU)
RenderingOGRE 2.x (CPU/GPU)RTX ray tracing (GPU)
Parallel Robots~10-201000+
Sensor RealismGoodPhotorealistic
Synthetic DataLimitedBuilt-in
RL TrainingExternal (Gym)Isaac Gym integrated
ROS 2 SupportNativeros2_bridge
CostFreeFree (with NVIDIA GPU)

When to Use Isaac Sim:

  • ✅ Training RL policies (massively parallel)
  • ✅ Generating synthetic training data
  • ✅ Testing perception algorithms (realistic sensors)
  • ✅ Large-scale multi-robot simulations

When to Use Gazebo:

  • ✅ CPU-only systems
  • ✅ Simple prototyping
  • ✅ Established ROS 1/2 workflows

Architecture Overview​

Omniverse Foundation​

Isaac Sim is built on NVIDIA Omniverse, a platform for 3D collaboration and simulation.

┌─────────────────────────────────────────────┐
│         Isaac Sim Application               │
│  (Robot simulation, sensors, RL training)   │
└──────────────┬──────────────────────────────┘
               │
┌──────────────▼──────────────────────────────┐
│         Omniverse Kit                       │
│  (Extensible app framework, USD stage)      │
└──────────────┬──────────────────────────────┘
               │
        ┌──────┴──────┬──────────┬──────────┐
        │             │          │          │
   ┌────▼────┐  ┌────▼────┐ ┌───▼───┐ ┌───▼───┐
   │ PhysX 5 │  │  RTX    │ │  USD  │ │ Python│
   │ (GPU)   │  │Renderer │ │Format │ │  API  │
   └─────────┘  └─────────┘ └───────┘ └───────┘

Key Components​

USD (Universal Scene Description)​

  • Open-source 3D scene format (Pixar)
  • Stores robots, environments, animations
  • Enables collaboration (multiple users editing same scene)

PhysX 5​

  • GPU-accelerated physics engine
  • Supports articulated bodies (humanoids)
  • Collision detection, contact dynamics
  • Tensor API for RL (direct GPU access)

RTX Renderer​

  • Real-time ray tracing
  • Physically-based materials
  • Realistic lighting and shadows
  • Sensor simulation (cameras, LiDAR)

Python API​

  • Full programmatic control
  • Custom extensions
  • RL environment creation

Installation and Setup​

System Requirements​

Minimum:

  • NVIDIA RTX GPU (2060 or higher)
  • Ubuntu 20.04/22.04 or Windows 10/11
  • 32 GB RAM
  • 50 GB disk space

Recommended:

  • NVIDIA RTX 3080/4080 or higher
  • 64 GB RAM
  • SSD storage

Installation​

# Chapter 3: Download Isaac Sim (requires NVIDIA account)
# Chapter 3: Visit: https://developer.nvidia.com/isaac-sim

# Chapter 3: Install via Omniverse Launcher
# Chapter 3: Install Omniverse Launcher
# Chapter 3: Install Isaac Sim from Exchange tab

# Chapter 3: Or install via pip (headless)
pip install isaacsim

# Chapter 3: Verify installation
python -c "from isaacsim import SimulationApp; print('Isaac Sim installed')"

Isaac Sim Interface​

GUI Components​

┌─────────────────────────────────────────────────┐
│  Viewport (3D scene with RTX rendering)         │
│                                                  │
│                                                  │
└─────────────────────────────────────────────────┘
┌──────────────┬──────────────┬──────────────────┐
│  Stage       │  Property    │  Content Browser │
│  (Scene tree)│  (Inspector) │  (Assets)        │
└──────────────┴──────────────┴──────────────────┘

Launching Isaac Sim​

# Chapter 3: GUI mode
~/.local/share/ov/pkg/isaac_sim-*/isaac-sim.sh

# Chapter 3: Headless mode (for servers)
~/.local/share/ov/pkg/isaac_sim-*/isaac-sim.sh --headless

# Chapter 3: Python script
python my_simulation.py

Creating a Simple Scene​

Python API Example​

from isaacsim import SimulationApp

# Chapter 3: Launch Isaac Sim
simulation_app = SimulationApp({"headless": False})

import omni
from omni.isaac.core import World
from omni.isaac.core.objects import DynamicCuboid
from omni.isaac.core.prims import RigidPrim

# Chapter 3: Create world
world = World(stage_units_in_meters=1.0)

# Chapter 3: Add ground plane
world.scene.add_default_ground_plane()

# Chapter 3: Add a cube
cube = world.scene.add(
    DynamicCuboid(
        prim_path="/World/Cube",
        name="my_cube",
        position=[0, 0, 1.0],
        size=0.5,
        color=[1.0, 0.0, 0.0]  # Red
    )
)

# Chapter 3: Reset world
world.reset()

# Chapter 3: Run simulation
for i in range(1000):
    world.step(render=True)  # Step physics and render

# Chapter 3: Cleanup
simulation_app.close()

GPU-Accelerated Physics​

PhysX 5 Features​

Articulated Bodies:

  • Humanoid robots with 20+ joints
  • Stable contact dynamics
  • GPU-accelerated forward kinematics

Parallel Simulation:

# Chapter 3: Simulate 1000 robots in parallel
num_envs = 1000
for env_id in range(num_envs):
    robot = create_robot(f"/World/Robot_{env_id}")
    # All robots simulated on GPU simultaneously

Performance:

  • CPU (Gazebo): ~10 robots @ 1x real-time
  • GPU (Isaac Sim): 1000+ robots @ 10x real-time

Photorealistic Rendering​

RTX Ray Tracing​

# Chapter 3: Enable RTX rendering
import carb
settings = carb.settings.get_settings()
settings.set("/rtx/rendermode", "PathTracing")
settings.set("/rtx/pathtracing/spp", 64)  # Samples per pixel

Features:

  • Real-time global illumination
  • Accurate shadows and reflections
  • Physically-based materials (PBR)

Sensor Simulation​

RGB Camera:

from omni.isaac.sensor import Camera

camera = Camera(
    prim_path="/World/Camera",
    position=[2, 0, 1],
    resolution=(1280, 720),
    frequency=30  # Hz
)

# Chapter 3: Get image
rgb_data = camera.get_rgba()

LiDAR:

from omni.isaac.range_sensor import LidarRtx

lidar = LidarRtx(
    prim_path="/World/Lidar",
    config="Velodyne_VLP16"  # Predefined config
)

# Chapter 3: Get point cloud
points = lidar.get_point_cloud_data()

ROS 2 Integration​

Isaac ROS 2 Bridge​

from omni.isaac.core.utils.extensions import enable_extension

# Chapter 3: Enable ROS 2 bridge
enable_extension("omni.isaac.ros2_bridge")

# Chapter 3: Publish joint states
from omni.isaac.core_nodes.scripts.utils import set_target_prims
set_target_prims(
    primPath="/World/Robot",
    targetPrimPaths=["/World/Robot/joint_state_publisher"]
)

Supported Messages:

  • sensor_msgs/JointState
  • sensor_msgs/Image
  • sensor_msgs/PointCloud2
  • geometry_msgs/Twist
  • nav_msgs/Odometry

Key Takeaways​

✅ Isaac Sim is GPU-accelerated, built on Omniverse

✅ PhysX 5 enables massively parallel physics (1000+ robots)

✅ RTX rendering provides photorealistic sensors

✅ USD format enables collaboration and asset reuse

✅ Python API for programmatic control

✅ ROS 2 bridge for integration with existing workflows

Reflection Questions​

  1. Why can Isaac Sim simulate 1000+ robots while Gazebo handles ~10-20?
  2. When would you choose Gazebo over Isaac Sim?
  3. How does RTX ray tracing improve sensor simulation?
  4. What is the role of USD in Isaac Sim?

Further Reading​

Previous Chapter: ← Chapter 3: Gazebo Simulation
Next Section: 4.2 Isaac Gym for RL →