Use ROCm on Radeon and Ryzen

Use ROCm on Radeon and Ryzen#

Unlock Local AI Development on Your AMD Hardware

Transform your AMD-powered system into a powerful and private machine learning workstation. With the latest ROCm™ software stack, you can now harness the full potential of high-end AMD Radeon™ GPUs and Ryzen™ APUs for your AI workflows on both Linux® and Windows®.

ROCm™ 7.2 supports the latest Radeon™ 9000 Series (RDNA™ 4) and select 7000 Series (RDNA™ 3) GPUs, and introduces support for Ryzen™ APUs, enabling cost-effective, local development and inference for researchers and engineers using Pytorch.

Expanded Platform Support#

Quickly see what’s supported on your system. ROCm™ 7.2 focuses on bringing PyTorch support to new platforms while maintaining robust support on our established Linux platform for Radeon GPUs.

Hardware	Operating System	Supported Frameworks
Radeon™ GPUs (9000 & select 7000 Series)	Linux®	PyTorch, TensorFlow, JAX, ONNX
Radeon™ GPUs (9000 & select 7000 Series)	Windows®	PyTorch
Ryzen™ APUs (AI Max 300 Series, select AI 400 Series & select AI 300 Series)	Linux®, Windows®	PyTorch

Your Local, Private AI Powerhouse#

Modern AI models demand significant computational power and memory. A local workstation equipped with a Radeon™ GPU, featuring up to 48GB of VRAM, offers a secure and economical alternative to relying solely on cloud-based solutions. Furthermore, expanded support to Ryzen™ APUs, with up to 128GB of shared memory, allow even laptop users to develop ML workflows securely and efficiently.

Leadership in Open-Source GPU Programming#

ROCm is the leading open-source software foundation for GPU programming. The same ROCm™ stack that powers your desktop development on RDNA™ architecture GPUs also supports AMD Instinct™ accelerators on CDNA™ architecture in the datacenter. This unified platform creates a seamless migration path, allowing you to develop applications locally and deploy them at scale with confidence. This maximizes GPU hardware investments, facilitating the development, testing, and deployment of AI workloads, GPU-accelerated HPC, scientific computing, CAD, and other applications.

As a primarily open-source ecosystem, ROCm™ gives you the freedom to inspect, customize, and tailor the software stack to your specific needs, backed by a collaborative community of developers.

ROCm™ Key Capabilities#

Frequent ROCm™ releases introduce major platform expansions and key software improvements.

New Platform Updates:

Windows: PyTorch on Windows updated with ROCm 7.2 on AMD Radeon graphics products and AMD Ryzen AI processors.
APU Linux Support: Initial PyTorch support and increased ROCm functionality for Ryzen™ APUs (Preview).
AMD ROCm™ 7.2 on Windows Subsystem for Linux 2 (WSL 2)
- The following has been added to WSL 2:
  - Support for ROCm 7.2
  - Support for Radeon RX 7700
- Recommended Windows driver:
  - AMD Software: Adrenalin Edition 26.1.1
- Find more information on ROCm on Radeon compatibility here and configuration of Windows Subsystem for Linux (WSL 2) here

Established Features & Framework Support (Radeon on Linux):

Pytorch and Tensorflow: Full and established support for both training and inference.
vLLM: Full support.
JAX: Support for JAX (inference only).
Llama.cpp: Supported for efficient inference.
FlashAttention-2: Backward pass enabled for more efficient training.
ONNX Runtime: Expanded support for INT8 and INT4 inference with MIGraphX.
Operating Systems: Support for Red Hat Enterprise Linux (RHEL) 10.1 (AMD Radeon graphics products only) as well as Ubuntu.
See the full support matrices here

Get Started#

Ready to build? Find all the resources you need at the links below.

ROCm Documentation for Instinct™: Visit the Official ROCm for Instinct Documentation Hub
Compatibility Matrices: Check Full Hardware/Software Compatibility
Linux Drivers: Download the Latest Radeon™ Software for Linux
Experiment with TheRock: Try the ROCm 7.10.0 Technology Preview Release
Join the Developer Discord: Join the AMD Developer Discord
AMD Strix Halo system optimization — ROCm Documentation: AMD Strix Halo system optimization — ROCm Documentation

Prerequisites to use ROCm on Radeon desktop GPUs for machine learning development#

Before starting with the installation, ensure that your system meets the necessary requirements such as supported hardware, a compatible operating system, and the recommended system configuration to ensure optimal performance and functionality.

See Compatibility matrices for more information.

Supported hardware#

Supported graphics processing units#

To successfully install ROCm™ for machine learning development, ensure that your system is operating on a Radeon™ Desktop GPU listed in the Compatibility matrices section.

Recommended memory#

The recommended memory to use ROCm on Radeon. These specifications are required for complex AI/ML workloads (for example, large language models):

64GB Main Memory
24GB GPU Video Memory

Note
AMD recommends having the same amount of system memory as video memory, as a minimum.

Minimum recommendations#

Minimum memory requirements to use ROCm on Radeon. Note that low system memory may cause issues running inference models.

16GB Main Memory
8GB GPU Video Memory

Important!
These are guidelines only. Note that minimum memory required will vary depending on workload.

Supported operating systems#

Ensure that your operating system is up-to-date to successfully install ROCm for machine learning development.

Refer to Compatibility matrices for up-to-date operating system compatibility.

Update Ubuntu® operating system#

Use the following commands to bring your OS up-to-date:

sudo apt-get update
sudo apt-get dist-upgrade

Recommended system configuration#

This section guides users on how to optimize system configuration for ROCm™ usage, ensuring smooth and performant ROCm operation.

PCIe atomics for PyTorch#

ROCm is an extension of HSA platform architecture, and shares queuing model, memory model, signaling and synchronization protocols.

Platform atomics are integral to perform queuing and signaling memory operations, where there may be multiple-writers across CPU and GPU agents.

For more details, see How ROCm uses PCIe atomics.

Disable iGPU#

The iGPU is non-essential for AI and ML workloads and not officially supported. Disable iGPU in SBIOS before proceeding to avoid unknown issues.

Alternatively, use environment variables to select the target GPU.

Here are examples to disable iGPU on some AMD motherboards:

Gigabyte™ X670 AORUS ELITE AX#

Enter BIOS

Path: Advanced → AMD CBS → NBIO Common Options → GFX Configuration → iGPU Configuration
Set iGPU to Disabled

ASUS Prime X670-P WIFI#

Enter BIOS

Path: Advanced → NB Configuration → Integrated Graphics
Set to Disabled

NOTE: This step only applies to AMD motherboards, no action is required for non-AMD motherboards.

There are no minimum motherboard hardware requirements.

Alternative option: Use environment variables to select target GPU#

An alternative option to disabling the iGPU is to use environment variable to select the GPU.

See GPU Isolation Techniques to specify the device indices you would like to expose to your application.

Additional package repositories#

For some distributions, the ROCm installation packages depend on packages that aren’t included in the default package repositories. These external repositories need to be sourced before installation. Use the following instructions specific to your distribution to add the necessary repositories.

Red Hat Enterprise Linux#

Add the EPEL repository for RHEL 10.

wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-10.noarch.rpm
sudo rpm -ivh epel-release-latest-10.noarch.rpm

Enable the CodeReady Linux Builder (CRB) repository.

In order to enable CRB, you may need to install dnf-plugin-config-manager first.
```
sudo dnf config-manager --enable codeready-builder-for-rhel-10-x86_64-rpms
```
Note

For more information, see Additional package repositories for Instinct GPUs.

Prerequisites to use ROCm on Ryzen APUs for machine learning development#

Before starting with the installation, ensure that your system meets the necessary requirements such as supported hardware, a compatible operating system, and the recommended system configuration to ensure optimal performance and functionality.

See Compatibility matrices for more information.

Supported Hardware#

Supported Graphics Processing Units#

To successfully install ROCm™ for machine learning development, ensure that your system is operating on a Ryzen™ APU listed in the Compatibility matrices section.

Recommended memory#

Recommended memory requirements to use ROCm on Ryzen. Note that low system memory may cause issues running inference models.

System Memory: 64GB+

Important! These are guidelines only. Note that minimum memory required will vary depending on workload.

Supported Operating Systems#

Ensure that your operating system is up-to-date to successfully install ROCm for machine learning development.

Refer to Compatibility matrices for up-to-date operating system compatibility.

Update Ubuntu® Operating System#

Use the following commands to bring your OS up-to-date:

sudo apt update
sudo apt upgrade

Linux How to guide - Use ROCm on Radeon and Ryzen#

This guide walks you through the various installation processes required to pair ROCm™ with the latest high-end AMD Radeon™ 7000 series desktop GPUs, and get started on a fully-functional environment for AI and ML development.

Install Radeon software for Linux with ROCm#

The ROCm™ Software Stack and other Radeon™ software for Linux components are installed using the amdgpu-install script to assist you in the installation of a coherent set of stack components.

Simplifies the installation of the AMDGPU stack by encapsulating the distribution specific package installation logic and by using command line options that allows you to specify the:
- Usecase of the AMDGPU stack to be installed (Graphics or Workstation)
- Combination of components (Pro stack, or user selection)
Performs post-install checks to verify whether the installation was performed successfully.
Installs the uninstallation script to allow you to remove the whole AMDGPU stack from the system by using a single command.

The script is provided by the installer package. See Compatibility matrices for support information.

Install AMD unified driver package repositories and installer script#

Select the applicable Ubuntu® version to download and install the amdgpu-install script on the system.

Ubuntu 24.04

Install kernel headers and development packages.

sudo apt install python3-setuptools python3-wheel

Enter the following commands to install the installer script for Ubuntu version 24.04.

sudo apt update
wget https://repo.radeon.com/amdgpu-install/7.2/ubuntu/noble/amdgpu-install_7.2.70200-1_all.deb
sudo apt install ./amdgpu-install_7.2.70200-1_all.deb

Ubuntu 22.04

Install kernel headers and development packages.

sudo apt install python3-setuptools python3-wheel

Enter the following commands to install the installer script for Ubuntu version 22.04.

sudo apt update
wget https://repo.radeon.com/amdgpu-install/7.2/ubuntu/jammy/amdgpu-install_7.2.70200-1_all.deb
sudo apt install ./amdgpu-install_7.2.70200-1_all.deb

RHEL 10.1

Note
See Additional package repositories for more information.

Install kernel headers and development packages.

sudo dnf install python3-setuptools python3-wheel

Install the amdgpu-install script on the system.

sudo yum update
wget https://repo.radeon.com/amdgpu-install/7.2/rhel/10.1/amdgpu-install-7.2.70200-1.el10.noarch.rpm
sudo yum install ./amdgpu-install-7.2.70200-1.el10.noarch.rpm

Install AMD unified kernel-mode GPU driver, ROCm, and graphics#

After the Unified Driver Deb Package repositories are installed, run the installer script with appropriate --usecase parameters to install the driver components.

AMD recommends installing the Graphics usecase by default. Only consider the alternative install option if you have an applicable Workstation usecase scenario.

Enter the following command to display a list of available usecases:

sudo amdgpu-install --list-usecase

Graphics usecase#

AMD recommends installing the Graphics usecase by default.

Run the following command to install open source graphics and ROCm.
```
amdgpu-install -y --usecase=graphics,rocm
```
Watch for output warning or errors indicating an unsuccessful driver installation.

Note:
The -y option installs non-interactively. This step may take several minutes, depending on internet connection and system speed.
Reboot the system.
```
sudo reboot
```

See Using the amdgpu-install script for more information.

Next, set Groups permissions.

Set Groups permissions#

Once the driver is installed, add any current user to the render and video groups to access GPU resources.

Reboot in order for group changes to take effect.

Add user to render and video groups#

Enter the following command to check groups in the system:
```
groups
```
Add user to the render and video group using the command:
```
sudo usermod -a -G render,video $LOGNAME
```
Reboot the system.
```
sudo reboot
```

See Setting Permissions for Groups for more information.

Post-install verification checks#

Run these post-installation checks to verify that the installation is complete:

Verify that the current user is added to the render and video groups.
```
groups
```
Expected result:
```
<username> adm cdrom sudo dip video plugdev render lpadmin lxd sambashare
```
<username> indicates the current user, and this result will vary in your environment.

Check if amdgpu kernel driver is installed.

dkms status

Expected result:

amdgpu/x.x.x-xxxxxxx.xx.xx, x.x.x-xx-generic, x86_64: installed

Check if the GPU is listed as an agent.

rocminfo

Expected result:

[...]
*******
Agent 2
*******
  Name:                    gfx1100
  Uuid:                    GPU-5ecee39292e80c37
  Marketing Name:          Radeon RX 7900 XTX
  Vendor Name:             AMD
  [...]
[...]

Check if the GPU is listed.

clinfo

Expected result:

[...]
  Platform Name:                 AMD Accelerated Parallel Processing
Number of devices:               1
  Device Type:                   CL_DEVICE_TYPE_GPU
  Vendor ID:                     1002h
  Board name:                    Radeon RX 7900 XTX
[...]

See Installing the all open usecase for additional troubleshooting tips.

Advanced install methods#

For advanced install methods, such as Multi-Version and Package Manager, refer to AMD GPU Install Script.

Uninstall ROCm#

Run the following command to uninstall the ROCm software stack and other Radeon software for Linux components:

sudo amdgpu-uninstall

Upgrade to newer versions of Radeon software for Linux#

The recommended method to upgrade is to uninstall, followed by an install.

Radeon Software for Linux does not support in-place upgrades.

Install PyTorch for ROCm#

Refer to this section for the recommended PyTorch via PIP installation method, as well as Docker-based installation.

PCIe atomics

ROCm is an extension of HSA platform architecture, and shares queuing model, memory model, signaling and synchronization protocols.

Platform atomics are integral to perform queuing and signaling memory operations, where there may be multiple-writers across CPU and GPU agents.

For more details, see How ROCm uses PCIe atomics.

Install methods#

AMD recommends the PIP install method to create a PyTorch environment when working with ROCm™ for machine learning development.

Option A: PyTorch via PIP installation#

AMD recommends the PIP install method to create a PyTorch environment when working with ROCm™ for machine learning development.

Check Pytorch.org for latest PIP install instructions and availability. See Compatibility matrices for support information.

Note
To install the following wheels, Python 3.12 must be set up.

Enter the following command to unpack and begin set up.
```
sudo apt install python3-pip -y
```
Enter this command to update the pip wheel.
```
pip3 install --upgrade pip wheel
```

Select the applicable Ubuntu version and enter the commands to install Torch and Torchvision for ROCm AMD GPU support.

This may take several minutes.

Important! AMD recommends proceeding with ROCm WHLs available at repo.radeon.com. The ROCm WHLs available at PyTorch.org are not tested extensively by AMD as the WHLs change regularly when the nightly builds are updated.

Important! When manually downloading WHLs from repo.radeon, ensure to select the compatible WHLs for specific Python versions.

See Compatibility matrices for support information.

Ubuntu 24.04

wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torch-2.9.1%2Brocm7.2.0.lw.git7e1940d4-cp312-cp312-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torchvision-0.24.0%2Brocm7.2.0.gitb919bd0c-cp312-cp312-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/triton-3.5.1%2Brocm7.2.0.gita272dfa8-cp312-cp312-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torchaudio-2.9.0%2Brocm7.2.0.gite3c6ee2b-cp312-cp312-linux_x86_64.whl
pip3 uninstall torch torchvision triton torchaudio
pip3 install torch-2.9.1+rocm7.2.0.lw.git7e1940d4-cp312-cp312-linux_x86_64.whl torchvision-0.24.0+rocm7.2.0.gitb919bd0c-cp312-cp312-linux_x86_64.whl torchaudio-2.9.0+rocm7.2.0.gite3c6ee2b-cp312-cp312-linux_x86_64.whl triton-3.5.1+rocm7.2.0.gita272dfa8-cp312-cp312-linux_x86_64.whl

Note

The --break-system-packages flag must be added when installing wheels for Python 3.12 in a non-virtual environment.

Ubuntu 22.04

wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torch-2.9.1%2Brocm7.2.0.lw.git7e1940d4-cp310-cp310-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torchvision-0.24.0%2Brocm7.2.0.gitb919bd0c-cp310-cp310-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/triton-3.5.1%2Brocm7.2.0.gita272dfa8-cp310-cp310-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torchaudio-2.9.0%2Brocm7.2.0.gite3c6ee2b-cp310-cp310-linux_x86_64.whl
pip3 uninstall torch torchvision triton torchaudio
pip3 install torch-2.9.1+rocm7.2.0.lw.git7e1940d4-cp310-cp310-linux_x86_64.whl torchvision-0.24.0+rocm7.2.0.gitb919bd0c-cp310-cp310-linux_x86_64.whl torchaudio-2.9.0+rocm7.2.0.gite3c6ee2b-cp310-cp310-linux_x86_64.whl triton-3.5.1+rocm7.2.0.gita272dfa8-cp310-cp310-linux_x86_64.whl

RHEL 10.1

wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torch-2.9.1%2Brocm7.2.0.lw.git7e1940d4-cp312-cp312-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torchvision-0.24.0%2Brocm7.2.0.gitb919bd0c-cp312-cp312-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/triton-3.5.1%2Brocm7.2.0.gita272dfa8-cp312-cp312-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torchaudio-2.9.0%2Brocm7.2.0.gite3c6ee2b-cp312-cp312-linux_x86_64.whl
pip3 uninstall torch torchvision triton torchaudio
pip3 install torch-2.9.1+rocm7.2.0.lw.git7e1940d4-cp312-cp312-linux_x86_64.whl torchvision-0.24.0+rocm7.2.0.gitb919bd0c-cp312-cp312-linux_x86_64.whl torchaudio-2.9.0+rocm7.2.0.gite3c6ee2b-cp312-cp312-linux_x86_64.whl triton-3.5.1+rocm7.2.0.gita272dfa8-cp312-cp312-linux_x86_64.whl

Note

The --break-system-packages flag must be added when installing wheels for Python 3.12 in a non-virtual environment.

Next, verify your PyTorch installation.

Option B: Docker installation#

Using Docker provides portability, and access to a prebuilt Docker container that has been rigorously tested within AMD. Docker also cuts down compilation time, and should perform as expected without installation issues.

Prerequisites to install PyTorch using Docker#

Docker for Ubuntu® must be installed.

To install Docker for Ubuntu, enter the following command:

sudo apt install docker.io

Use Docker image with pre-installed PyTorch#

Follow these steps to install using a Docker image.

Note
The latest version of Python module numpy v2.0 is incompatible with the torch wheels for this version. Downgrade to an older version is required.
Example: pip3 install numpy==1.26.4

Select the applicable Ubuntu version and enter the following command to pull the public PyTorch Docker image.

Optional: You can also download a specific and supported configuration with different user-space ROCm versions, PyTorch versions, and supported operating systems.

Refer to hub.docker.com/r/rocm/pytorch to download the PyTorch Docker image.
Ubuntu 24.04
sudo docker pull rocm/pytorch:rocm7.2_ubuntu24.04_py3.12_pytorch_release_2.9.1
Ubuntu 22.04
sudo docker pull rocm/pytorch:rocm7.2_ubuntu22.04_py3.10_pytorch_release_2.9.1

Select the applicable Ubuntu version and start a Docker container using the downloaded image.

Ubuntu 24.04

sudo docker run -it \
  --cap-add=SYS_PTRACE \
  --security-opt seccomp=unconfined \
  --device=/dev/kfd \
  --device=/dev/dri \
  --group-add video \
  --ipc=host \
  --shm-size 8G \
  rocm/pytorch:rocm7.2_ubuntu24.04_py3.12_pytorch_release_2.9.1

Ubuntu 22.04

sudo docker run -it \
  --cap-add=SYS_PTRACE \
  --security-opt seccomp=unconfined \
  --device=/dev/kfd \
  --device=/dev/dri \
  --group-add video \
  --ipc=host \
  --shm-size 8G \
  rocm/pytorch:rocm7.2_ubuntu22.04_py3.10_pytorch_release_2.9.1

This will automatically download the image if it does not exist on the host. You can also pass the -v argument to mount any data directories from the host onto the container.

Next, verify the PyTorch installation.

See PyTorch Installation for ROCm for more information.

Verify PyTorch installation#

Confirm if PyTorch is correctly installed.

Verify if Pytorch is installed and detecting the GPU compute device.

python3 -c 'import torch' 2> /dev/null && echo 'Success' || echo 'Failure'

Expected result:

Success

Enter command to test if the GPU is available.

python3 -c 'import torch; print(torch.cuda.is_available())'

Expected result:

True

Enter command to display installed GPU device name.

python3 -c "import torch; print(f'device name [0]:', torch.cuda.get_device_name(0))"

Expected result: Example: device name [0]: Radeon RX 7900 XTX

device name [0]: <Supported AMD GPU>

Enter command to display component information within the current PyTorch environment.

python3 -m torch.utils.collect_env

Expected result:

PyTorch version
 
ROCM used to build PyTorch
 
OS
 
Is CUDA available
 
GPU model and configuration
 
HIP runtime version
 
MIOpen runtime version

Environment set-up is complete, and the system is ready for use with PyTorch to work with machine learning models, and algorithms.

Install ONNX Runtime for Radeon GPUs#

Overview#

Ensure that the following prerequisite installations are successful before proceeding to install ONNX Runtime for use with ROCm™ on Radeon™ GPUs.

Prerequisites#

Radeon Software for Linux (with ROCm) is installed.
MIGraphX is installed. This enables ONNX Runtime to build the correct MIGraphX Execution Provider (EP).
The half library is installed. See Verify if MIGraphX is installed with the half library.
If the prerequisite installations are successful, proceed to install ONNX Runtime.

NOTE Unless adding custom features, use the pre-built Python wheel files provided in the PIP installation method.

Verify MIGraphX installation#

Verify if MIGraphX is installed with the half library

$ dpkg -l | grep migraphx
$ dpkg -l | grep half

Expected result:

root@aus-navi3x-02:/workspace/AMDMIGraphX# dpkg -l | grep migraphx
ii  migraphx                      2.9.0                             amd64        AMD's graph optimizer
ii  migraphx-dev                  2.9.0                             amd64        AMD's graph optimizer
ii  migraphx-tests                2.9.0                             amd64        AMD's graph opt


$ dpkg -l | grep half
ii  half                          1.12.0.60000-91~20.04             amd64        HALF-PRECISION FLOATING POINT LIBRARY

Note
Versions may vary between ROCm builds and installed versions of MIGraphX, but the desired result is the same.

The half library should come packaged with MIGraphX. If not, it can be installed with the following command.
```
sudo apt install half
```
Perform a simple inference with MIGraphX to verify the installation.
```
/opt/rocm-7.2.0/bin/migraphx-driver perf --test
```

Install ONNX Runtime#

Important!

Use the provided pre-built Python wheel files from the PIP installation method, unless adding custom features.

The wheel file contains the MIGraphX and ROCm Execution Providers (EP). Refer to Install MIGraphX for ONNX RT for more information.

Refer to ONNX Runtime Documentation for additional information on ONNX Runtime topics.

See ONNX Runtime Tutorials to try out real applications and tutorials on how to get started.

Option A: PIP install (Recommended)

Option A: ONNX Runtime install via PIP installation method (Recommended)

AMD recommends the PIP install method to create an ONNX Runtime environment when working with ROCm for machine learning development.

Note
The latest version of Python module numpy v2.0 is incompatible with the ONNX Runtime wheels for this version. Downgrade to an older version is required.
Example: pip3 install numpy==1.26.4

To install via PIP,

Enter this command to download and install the ONNX Runtime wheel.

pip3 uninstall onnxruntime-migraphx
pip3 install onnxruntime-migraphx -f https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/

Next, verify your ONNX Runtime installation.

Option B: Build from source (Advanced)

Option B: Build from source for your environment, followed by local wheel file installation (Advanced)

Use this method for advanced customization usecases. This requires the user install the desired ROCm and MIGraphX versions, and creation of softlink prior to starting the build.

NOTE The build time typically takes ~45 minutes.

Prerequisites to build ONNX from source

Radeon Software for Linux (with ROCm) is installed
MigraphX is installed
Softlink is created
To create a softlink for /opt/rocm, enter the following command:
language = bash linenumbers = true ln -s /opt/rocm* /opt/rocm

To build from source,

Clone the ONNX Runtime repository into the root directory.

cd /
git clone https://github.com/microsoft/onnxruntime.git

Git clone AMDMIGraphX into the home folder.

cd ~
git clone https://github.com/ROCm/AMDMIGraphX.git

Create a docker image for MIGraphX.
Note
- Refer to AMDMIGraphX Github for up-to-date ONNX Runtime and MIGraphX dependencies.
- MIGraphX can still be built or installed from apt.
- For MIGraphX package builds via RBuild, refer to these MIGraphx Github instructions to build within the docker container environment.
- For MIGraphX package builds via CMake, refer to these MIGraphx Github instructions to build within the docker container environment.
- Use the groups command to ensure that the user is part of the video, render, and docker groups in Linux to run the docker container.
```
groups
tthemist@aus-navi3x-02 ~/groups
tthemist sudo video render docker
```
- Run the following for a simple MIGraphX apt install:
```
cd AMDMIGraphX
docker build -t migraphx .
docker run --device='/dev/kfd' --device='/dev/dri' -v=`pwd`:/code/AMDMIGraphX -v /onnxruntime:/onnxruntime -w /code/AMDMIGraphX --group-add video -it migraphx
apt install migraphx migraphx-dev half
```

Run rocm-smi to ensure that ROCm is installed and detects the supported GPU(s).

$ rocm-smi

Expected result:

======================================= ROCm System Management Interface =======================================
================================================= Concise Info =================================================
Device  [Model : Revision]    Temp    Power  Partitions      SCLK     MCLK   Fan     Perf  PwrCap  VRAM%  GPU% 
        Name (20 chars)       (Edge)  (Avg)  (Mem, Compute)                                                    
================================================================================================================
0       [0x0e0d : 0x00]       32.0°C  73.0W  N/A, N/A        1526Mhz  96Mhz  31.76%  auto  241.0W    0%   50%  
        0x7448                                                                                                 
================================================================================================================
============================================= End of ROCm SMI Log ==============================================

Configure Git to treat all directories as safe to use and run the build script.
```
cd AMDMIGraphX
git config --global --add safe.directory "*"
tools/build_and_test_onnxrt.sh
```
This builds ONNX Runtime and adds ROCm and MIGraphX EP support to the ONNX Runtime interface and requires multiple external repo pieces be checked out automatically prior to the build.

Install ONNX Runtime once MIGraphX is built.

$ pip3 install /onnxruntime/build/Linux/Release/dist/*.whl

Next, verify your ONNX Runtime installation.

Verify ONNX Runtime installation#

Verify that the install works correctly by performing a simple inference with MIGraphX.

python3 -c "import onnxruntime as ort; print(ort.get_available_providers())"

Expected result: The following EPs are displayed.

>>> import onnxruntime as ort
>>> ort.get_available_providers()
['MIGraphXExecutionProvider', 'CPUExecutionProvider']

This indicates that the MIGraphXExecutionProvider and ROCMExecutionProvider are now running on the system, and the proper ONNX Runtime package has been installed.

Installation is complete and ONNX Runtime is available through the Python interface library, as well as scripts that invoke ONNX Runtime inference sessions.

For more information on the ONNX Runtime Python library, refer to Get started with ONNX Runtime in Python.

Install TensorFlow for ROCm#

TensorFlow is an open-source library for solving machine-learning, deep-learning, and artificial-intelligence problems. It can be used to solve many problems across different sectors and industries but primarily focuses on training and inference in neural networks. It is one of the most popular and in-demand frameworks and is very active in open source contribution and development.

Important!
ROCm on Radeon scripts uses Keras 2, but newest wheels uses Keras 3 by default.

A manual install of the tf-keras package is required to enable Keras 2 on TensorFlow.

Use the following command to install the ROCm compatible TensorFlow wheel.
`pip install tf-keras --no-deps`

Additional information As of ROCm 6.1, tensorflow-rocm packages are found at https://repo.radeon.com/rocm/manylinux. Prior to ROCm 6.1, packages were found at https://pypi.org/project/tensorflow-rocm.
Refer to the following version support matrix:

ROCm version

TensorFlow version

6.1.x

2.13.1, 2.14.0, 2.15.0

6.0.x

2.12, 2.13.1, 2.14.0

Pre-requisites#

Radeon software for Linux (with ROCm) must be installed.
MIGraphX must be installed for TensorFlow to build the correct mig execution provider.

PIP installation#

Use the PIP install method to create a TensorFlow environment when working with ROCm for machine learning development.

Library Compatibility
When installing TensorFlow, it is essential to ensure that any additional TensorFlow-related libraries, or dependencies are compatible with the version of TensorFlow that you are using.

TensorFlow libraries (such as tensorflow-hub, etc.) often have specific version requirements that depend on the main TensorFlow version.

Be aware that installing or upgrading TensorFlow libraries may inadvertently replace the ROCm-supported TensorFlow package with a non-ROCm supported version.

To avoid this, ensure that you are explicitly specifying the ROCm-compatible version during installation.

To install TensorFlow,

Ubuntu 24.04

Download and install the TensorFlow wheel.

Note

The --break-system-packages flag must be added when installing wheels for Python 3.12 in a non-virtual environment.

pip3 uninstall tensorflow-rocm
pip3 install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/tensorflow_rocm-2.20.0.dev0%2Bselfbuilt-cp312-cp312-manylinux_2_28_x86_64.whl

Ubuntu 22.04

Download and install the TensorFlow wheel.

pip3 uninstall tensorflow-rocm
pip3 install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/tensorflow_rocm-2.20.0.dev0%2Bselfbuilt-cp310-cp310-manylinux_2_28_x86_64.whl

RHEL 10.1

Download and install the TensorFlow wheel.

Note

The --break-system-packages flag must be added when installing wheels for Python 3.12 in a non-virtual environment.

pip3 uninstall tensorflow-rocm
pip3 install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/tensorflow_rocm-2.20.0.dev0%2Bselfbuilt-cp312-cp312-manylinux_2_28_x86_64.whl

Next, verify your TensorFlow installation.

Verify TensorFlow installation#

To test the TensorFlow installation, run the container image as specified in the previous section Installing TensorFlow. Ensure you have access to the Python shell in the Docker container.

python3 -c 'import tensorflow' 2> /dev/null && echo 'Success' || echo 'Failure'

Next, run basic TensorFlow example.

Run basic TensorFlow example#

The TensorFlow examples repository provides basic examples that exercise the framework’s functionality.

The MNIST database is a collection of handwritten digits that may be used to train a Convolutional Neural Network for handwriting recognition.

This dataset is included with your TensorFlow installation.

Run the following sample code to load the MNIST dataset, then train and evaluate it.

import tensorflow as tf
print("TensorFlow version:", tf.__version__)
mnist = tf.keras.datasets.mnist

(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28, 28)),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(10)
])
predictions = model(x_train[:1]).numpy()
tf.nn.softmax(predictions).numpy()
loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
loss_fn(y_train[:1], predictions).numpy()
model.compile(optimizer='adam',
              loss=loss_fn,
              metrics=['accuracy'])
model.fit(x_train, y_train, epochs=5)
model.evaluate(x_test,  y_test, verbose=2)

If successful, you should see the following output indicating the image classifier is now trained to around 98% accuracy on this dataset.

Environment set-up is complete, and the system is ready for use with TensorFlow to work with machine learning models, and algorithms.

Install Triton for ROCm#

OpenAI has developed a powerful GPU focused programming language and compiler called Triton that works seamlessly with AMD GPUs. The goal of Triton is to enable AI engineers and scientists to write high-performant GPU code with minimal expertise.

Triton kernels are performant because of their blocked program representation, allowing them to be compiled into highly optimized binary code. Triton also leverages Python for kernel development, making it both familiar and accessible.

The kernels can be compiled by declaring the triton.jit python decorator before the kernel.

Pre-requisites#

Compatible AMD GPU
Linux and ROCm 5.7+ is installed

See Compatibility matrices for support information.

Install libraries#

If ROCm 6.0 and the latest version of PyTorch is not installed, the required libraries must first be installed. However, if you encounter issues running any of the commands, we recommend updating with the nightly wheels. This will also install the version of Triton that is compatible with PyTorch for ROCm.

Enter the following command to install the libraries.

pip install matplotlib pandas -q
pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm6.0/ -q

Enter the following command to import the libraries.

import torch
import triton
import triton.language as tl

Now, a Triton kernel that approximates the GELU (Gaussian Error Linear Unit) kernel using tanh can be developed.

For more information on how to develop a kernel for GELU and benchmark its performance with its PyTorch analogues, see Developing Triton Kernels on AMD GPUs.

Install JAX for ROCm#

This directory provides setup instructions and necessary files to build, test, and run JAX with ROCm support using a PIP or Docker install, suitable for both runtime and CI workflows.

Note
These instructions are for JAX installation on Radeon GPUs.
To install ROCm on Instinct GPUs, refer to ROCm Instinct documentation.

Install JAX#

Follow these instructions to install JAX via PIP install or Docker install.

PIP installation#

Follow these instructions to install JAX via PIP.

Important
The packages must be installed in the following order:

Install pjrt wheel.

Install plugin wheel.

Install jaxlib wheel.

Install jax wheel.

Ubuntu 24.04

Install JAX for Ubuntu 24.04.

Uninstall previous version

pip3 uninstall -y jax-rocm7-pjrt jax-rocm7-plugin jaxlib jax

Install the pjrt wheel.

pip install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/jax_rocm7_pjrt-0.8.0%2Brocm7.2.0-py3-none-manylinux_2_28_x86_64.whl

Install the plugin wheel.

pip install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/jax_rocm7_plugin-0.8.0%2Brocm7.2.0-cp312-cp312-manylinux_2_28_x86_64.whl

Install the jaxlib wheel.
```
pip install jaxlib==0.8.0
```
Install the jax wheel.
```
pip install jax==0.8.0
```

Ubuntu 22.04

Install JAX for Ubuntu 22.04.

Note
Latest JAX no longer supports for Python 3.10. Install Python 3.11 to use JAX with Ubuntu 22.04.

Uninstall previous version

pip3 uninstall -y jax-rocm7-pjrt jax-rocm7-plugin jaxlib jax

Install the pjrt wheel.

pip install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/jax_rocm7_pjrt-0.8.0%2Brocm7.2.0-py3-none-manylinux_2_28_x86_64.whl

Install the plugin wheel.

pip install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/jax_rocm7_plugin-0.8.0%2Brocm7.2.0-cp311-cp311-manylinux_2_28_x86_64.whl

Install the jaxlib wheel.
```
pip install jaxlib==0.8.0
```
Install the jax wheel.
```
pip install jax==0.8.0
```

RHEL 10.1

Install JAX for RHEL 10.1.

Uninstall previous version

pip3 uninstall -y jax-rocm7-pjrt jax-rocm7-plugin jaxlib jax

Install the pjrt wheel.

pip install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/jax_rocm7_pjrt-0.8.0%2Brocm7.2.0-py3-none-manylinux_2_28_x86_64.whl

Install the plugin wheel.

pip install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/jax_rocm7_plugin-0.8.0%2Brocm7.2.0-cp312-cp312-manylinux_2_28_x86_64.whl

Install the jaxlib wheel.
```
pip install jaxlib==0.8.0
```
Install the jax wheel.
```
pip install jax==0.8.0
```
Recommended for RHEL distros: Install the gcc-gfortran package (for libgfortran.so).
```
sudo dnf install gcc-gfortran
```

Docker installation#

The ROCm JAX team provides prebuilt Docker images, which is the simplest way to use JAX on ROCm. These images are available on Docker Hub and come with JAX configured for ROCm.

Note
If issues occur while installing python packages within the docker, add the pip install parameter --break-system-packages command.

Ubuntu 24.04

Install JAX for Ubuntu 24.04.

To pull the latest ROCm JAX Docker image, run:
```
docker pull rocm/jax:rocm7.2-jax0.8.0-py3.12
```
Note
For specific versions of JAX, review the periodically pushed Docker images at ROCm JAX Community on Docker Hub.

Additional Docker images are available at ROCm JAX on Docker Hub. These contain the latest ROCm version but might use an older version of JAX.
Once the image is downloaded, launch a container using the following command:
```
docker run -it -d --network=host --device=/dev/kfd --device=/dev/dri --ipc=host --shm-size 64G \
--group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -v $(pwd):/jax_dir \
--name rocm_jax rocm/jax-community:latest /bin/bash

docker attach rocm_jax
```
Note
The –shm-size parameter allocates shared memory for the container. Adjust it based on your system’s resources if needed. Replace $(pwd) with the absolute path to the directory you want to mount inside the container. If you prefer to use rocm/jax, remember to replace rocm/jax-community with rocm/jax.

Ubuntu 22.04

Install JAX for Ubuntu 22.04.

To pull the latest ROCm JAX Docker image, run:
```
docker pull rocm/jax:rocm7.2-jax0.8.0-py3.11
```
Note
For specific versions of JAX, review the periodically pushed Docker images at ROCm JAX Community on Docker Hub.

Additional Docker images are available at ROCm JAX on Docker Hub. These contain the latest ROCm version but might use an older version of JAX.
Once the image is downloaded, launch a container using the following command:
```
docker run -it -d --network=host --device=/dev/kfd --device=/dev/dri --ipc=host --shm-size 64G \
--group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -v $(pwd):/jax_dir \
--name rocm_jax rocm/jax-community:latest /bin/bash

docker attach rocm_jax
```
Note
The –shm-size parameter allocates shared memory for the container. Adjust it based on your system’s resources if needed. Replace $(pwd) with the absolute path to the directory you want to mount inside the container. If you prefer to use rocm/jax, remember to replace rocm/jax-community with rocm/jax.

RHEL 10

Install JAX for RHEL 10.

To pull the latest ROCm JAX Docker image, run:
```
ddocker pull rocm/jax:rocm7.2-jax0.8.0-py3.12
```
Note
For specific versions of JAX, review the periodically pushed Docker images at ROCm JAX Community on Docker Hub.

Additional Docker images are available at ROCm JAX on Docker Hub. These contain the latest ROCm version but might use an older version of JAX.
Once the image is downloaded, launch a container using the following command:
```
docker run -it -d --network=host --device=/dev/kfd --device=/dev/dri --ipc=host --shm-size 64G \
--group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -v $(pwd):/jax_dir \
--name rocm_jax rocm/jax-community:latest /bin/bash

docker attach rocm_jax
```
Note
The –shm-size parameter allocates shared memory for the container. Adjust it based on your system’s resources if needed. Replace $(pwd) with the absolute path to the directory you want to mount inside the container. If you prefer to use rocm/jax, remember to replace rocm/jax-community with rocm/jax.

Verify installation#

Refer to Testing your JAX installation with ROCm for verification steps.

Install MIGraphX for Radeon GPUs#

MIGraphX is AMD’s graph inference engine that accelerates machine learning model inference, and can be used to accelerate workloads within the Torch MIGraphX and ONNX Runtime backend frameworks.

Torch-MIGraphX, which integrates MIGraphX with PyTorch
MIGraphX for ONNX Runtime backend, which integrates MIGraphX with ONNX

ONNX Runtime can be driven by either the ROCm™ Execution Provider (EP) or MIGraphX EP

Introduction to MIGraphX#

MIGraphX is a graph optimizer that accelerates the inference for deep learning models. It provides C++ and Python APIs that are integrated within frameworks like Torch MIGraphX, ONNX Runtime, and other user solutions. The following process summarizes the procedures that occur under-the-hood during the optimization and real-time compilation process.

MIGraphX accelerates the Machine Learning models by leveraging several graph-level transformations and optimizations. These optimizations include:

Operator fusion
Arithmetic simplifications
Dead-code elimination
Common subexpression elimination (CSE)
Constant propagation

When the aforementioned optimizations are applied, MIGraphX emits code for the AMD GPU by calling to MIOpen, rocBLAS, or creating HIP kernels for a particular operator. MIGraphX can also target CPUs using DNNL or ZenDNN libraries.

For more information on how to install MIGraphX, refer to AMD MIGraphX Github.

Prerequisites#

Radeon™ Software for Linux (with ROCm) is installed

Install MIGraphX#

Install MIGraphX on your computer. Once the install is completed and verified, proceed to install Torch-MIGraphX or MIGraphX for ONNX Runtime.

Run the following command to install MIGraphX:

$ sudo apt install migraphx

Next, proceed to install Torch-MIGraphX or MIGraphX for ONNX Runtime as applicable.

Install Torch-MIGraphX#

Install Torch-MIGraphX using the Docker installation method, or build from source.

Option A: Docker installation

Using Docker provides portability, and access to a prebuilt Docker container that has been rigorously tested within AMD. Docker also cuts down compilation time, and should perform as expected without installation issues.

Clone the torch_migraphx repository.

git clone https://github.com/ROCmSoftwarePlatform/torch_migraphx.git

Change directory to torch-migraphx.
```
cd torch_migraphx/
```
Build image using the provided script.
```
sudo ./build_image.sh
```

Run container.

sudo docker run -it --network=host --device=/dev/kfd --device=/dev/dri --group-add=video --ipc=host --cap-add=SYS_PTRACE --security-opt seccomp=unconfined torch_migraphx

Next, verify the Torch-MIGraphX installation.

Option B: Build from source

To build from source in a custom environment, refer to the torch_migraphx repository for build steps.

Next, verify the Torch-MIGraphX installation.

Verify Torch-MIGraphX installation#

Verify if the Torch-MIGraphX installation is successful.

Verify if torch_migraphx can be imported as a Python module.

python3 -c 'import torch_migraphx' 2> /dev/null && echo 'Success' || echo 'Failure'

Run unit tests.
```
pytest ./torch_migraphx/tests
```

Installation is complete and the system is able to run PyTorch through the python interface library, and scripts that invoke PyTorch inference sessions.

Install and verify MIGraphX for ONNX Runtime#

See Install ONNX Runtime for Radeon GPUs for MIGraphX for ONNX Runtime installation and verification instructions.

mGPU setup and configuration#

Hardware and software considerations#

Refer to the following hardware and software considerations to ensure optimal performance.

Note
Current mGPU enablement supports up to a x4 configuration.

Hardware considerations#

PCIe® slots
AMD recommends a system with multiple x16 (Gen 4) slots, with optimal performance achieved by provision of a 1:1 ratio between the number of x16 slots and the number of GPUs used.

Note
Functionality is maintained in the instance where only one x16 slot is available, at the cost of some performance.
mGPU power setup
MultiGPU configurations require adequate amounts of power for all the components required.
Consult AMD Radeon™ RX or AMD Radeon™ PRO for GPU specifications and graphics card power requirements.

Software considerations#

There are no differences in software requirements between single-GPU and multi-GPU usage.

mGPU configuration by framework#

Note
PyTorch, ONNX, and Tensorflow may have additional guidelines regarding mGPU configuration. Refer to official mGPU support documentation of the applicable framework for more information.

PyTorch#

Warning

It is recommended to use DistributedDataParallel, instead of DataParallel to do multi-GPU training, even if there is only a single node. See: Use nn.parallel.DistributedDataParallel instead of multiprocessing or nn.DataParallel and Distributed Data Parallel.

mGPU known issues and limitations#

AMD has identified common errors when running ROCm™ on Radeon™ multi-GPU configuration at this time, along with the applicable recommendations.

IOMMU limitations and guidance#

For any issues with application hangs, or problems running a workload when running on a system with multiple GPUs, see Issue #5: Application hangs on Multi-GPU systems.

Windows Subsystem for Linux (WSL) support#

Microsoft does not currently support mGPU setup in WSL.

Simultaneous parallel compute workloads#

Radeon GPUs does not support large amounts of simultaneous, parallel workloads. It is not recommended to exceed 2 simultaneous compute workloads, with the assumption that workloads are running alongside a graphics environment (eg: Linux desktop).

Recommended multi-GPU system configuration#

PCIe slots connected to the GPU must have identical PCIe lane width or bifurcation settings, and support PCIe 3.0 Atomics.

Refer to How ROCm uses PCIe Atomics for more information.

Example:

✓ - GPU0 PCIe x16 connection + GPU1 PCIe x16 connection

✓ - GPU0 PCIe x8 connection + GPU1 PCIe x8 connection

X - GPU0 PCIe x16 connection + GPU1 PCIe x8 connection

Important!

Only use PCIe slots connected by the CPU and to avoid PCIe slots connected via chipset. Refer to product-specific motherboard documentation for PCIe electrical configuration.

Ensure the system Power Supply Unit (PSU) has sufficient wattage to support multiple GPUs.

GPU isolation techniques#

For more information, see GPU isolation techniques.

PCIe atomic operations#

Some consumer grade motherboards may only support the first PCIe slot. For unexpected issues, see How ROCm uses PCIe atomics.

Errors due to GPU and PCIe configuration#

When using two AMD Radeon 7900XTX GPUs, the following HIP error is observed when running PyTorch micro-benchmarking if any one of the two GPUs are connected to a non-CPU PCIe slot (PCIe on chipset):

RuntimeError: HIP error: the operation cannot be performed in the present state
HIP kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing HIP_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_HIP_DSA` to enable device-side assertions.

Potential GPU reset with some mixed graphics and compute workloads#

Working with certain mixed graphics and compute workloads may result in a GPU reset on Radeon GPUs.

Currently identified scenarios include:

Running multiple ML workloads simultaneously while using the desktop
Running ML workloads while simultaneously using Blender/HIP

Accelerate Library & Multiple GPU devices#

Avoid unexpected out of memory errors when working with large models & inference

Refer to the following sections on infer_auto_device_map() to explicitly define max memory for each GPU device, when working with accelerate library & multiple GPU devices.

How to guide - Use ROCm on Radeon and Ryzen#

This guide walks you through the various installation processes required to pair ROCm™ with the latest high-end AMD Radeon™ 7000 series desktop GPUs, and get started on a fully-functional environment for AI and ML development.

PyTorch via PIP installation#

AMD recommends the PIP install method to create a PyTorch environment when working with ROCm™ for machine learning development.

Note
To install the following wheels, Python 3.12 must be installed.

Prerequisites#

For the 7.2 PyTorch on Windows release, the 26.1.1 graphics driver must be installed.

Install PyTorch via PIP#

Enter the commands to set up ROCm environment.

CMD Prompt

pip install --no-cache-dir ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_core-7.2.0.dev0-py3-none-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_devel-7.2.0.dev0-py3-none-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_libraries_custom-7.2.0.dev0-py3-none-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm-7.2.0.dev0.tar.gz

PowerShell

pip install --no-cache-dir `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_core-7.2.0.dev0-py3-none-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_devel-7.2.0.dev0-py3-none-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_libraries_custom-7.2.0.dev0-py3-none-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm-7.2.0.dev0.tar.gz

Enter the commands to install torch, torchvision and torchaudio for ROCm AMD GPU support.

Note
This may take several minutes. See Compatibility matrices for support information.

CMD Prompt

pip install --no-cache-dir ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torch-2.9.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torchaudio-2.9.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torchvision-0.24.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl

PowerShell

pip install --no-cache-dir `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torch-2.9.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torchaudio-2.9.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torchvision-0.24.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl

Verify PyTorch installation#

Confirm if PyTorch is correctly installed.

Verify if Pytorch is installed and detecting the GPU compute device.
```
python -c "import torch" 2>nul && echo Success || echo Failure
```
Expected result:
```
Success
```

Enter command to test if the GPU is available.

python -c "import torch; print(torch.cuda.is_available())"

Expected result:

True

Enter command to display installed GPU device name.

python -c "import torch; print(f'device name [0]:', torch.cuda.get_device_name(0))"

Example result: device name [0]: Radeon RX 7900 XTX

device name [0]: <Supported AMD GPU>

Enter command to display component information within the current PyTorch environment.

python -m torch.utils.collect_env

Example result:

PyTorch version: 2.9.1+rocmsdk20260116
Is debug build: False
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: 7.2.26024-f6f897bd3d

OS: Microsoft Windows 11 Pro (10.0.26100 64-bit)
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.12.10 (tags/v3.12.10:0cc8128, Apr  8 2025, 12:21:36) [MSC v.1943 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-11-10.0.26100-SP0
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to:
GPU models and configuration: AMD Radeon PRO W7900 (gfx1100)
Nvidia driver version: Could not collect
cuDNN version: Could not collect
Is XPU available: False
HIP runtime version: 7.2.26024
MIOpen runtime version: 3.5.1
Is XNNPACK available: True

Environment set-up is complete, and the system is ready for use with PyTorch to work with machine learning models, and algorithms.

Install HIP SDK for Radeon GPUs#

The HIP SDK for Windows brings a subset of the ROCm platform to Windows. It provides APIs and tooling to leverage the computational power of accelerators and GPUs to create high-performance, portable applications using HIP.

For HIP SDK installation instructions, component support, system requirements and limitations, refer to HIP SDK installation for Windows.

Linux How to guide - Use ROCm on Ryzen#

This guide walks you through the various installation processes required to pair ROCm™ with the latest high-end AMD Ryzen™ APUs, and get started on a fully-functional environment for AI and ML development.

Install Ryzen Software for Linux with ROCm#

The ROCm™ Software Stack and other Ryzen™ software for Linux components are installed using the amdgpu-install script to assist you in the installation of a coherent set of stack components.

Simplifies the installation of the AMDGPU stack by encapsulating the distribution-specific package installation logic and by using command line options.
Performs post-install checks to verify whether the installation was performed successfully.
Installs the uninstallation script to allow you to remove the ROCm stack from the system by using a single command.

The script is provided by the installer package. See Compatibility matrices for support information.

Prepare the system#

For ROCm on Ryzen, it is required to operate on the 6.14-1018 OEM kernel or newer.

To install the kernel, please run the following command:

sudo apt update && sudo apt install linux-oem-24.04c

Once installation is complete, please reboot your system and ensure that you’ve booted into the correct kernel:
```
uname -r
```
Note
This returns a 6.14-1018 or newer based string.
Ensure that the system is up to date:
```
sudo apt upgrade -y
```

Install AMD Unified Driver Package Repositories and Installer Script#

Select the applicable Ubuntu® version to download and install the amdgpu-install script on the system.

Ubuntu 24.04

Enter the following commands to install the installer script for Ubuntu® version 24.04:

sudo apt update
wget https://repo.radeon.com/amdgpu-install/7.2/ubuntu/noble/amdgpu-install_7.2.70200-1_all.deb
sudo apt install ./amdgpu-install_7.2.70200-1_all.deb

Install AMD ROCm package#

Run the installer script with appropriate --usecase parameters to install the components once the Unified Driver Deb Package repositories are installed.

Set up ROCm usecase#

The --no-dkms parameter must be passed, as inbox drivers are required for ROCm on Ryzen.

Run the following command to install ROCm:

amdgpu-install -y --usecase=rocm --no-dkms

Note:
The -y option installs non-interactively. This step may take several minutes, depending on internet connection and system speed.
Look out for output warning or errors that indicate an unsuccessful installation.

Uninstall Dkms:
In the event that dkms is unintentionally installed, run the following command to remove:
sudo apt autoremove amdgpu-dkms dkms
Confirm that the removal was successful using dkms status (will return “blank” if removal is successful).

See Using the amdgpu-install script for more information.

Next, set Groups permissions.

Set Groups permissions#

Once the driver is installed, add any current user to the render and video groups to access GPU resources.

Reboot for group changes to take effect.

Add User to Render and Video Groups Enter the following command to check groups in the system:
```
groups
```
Add the user to the render and video group using the command:
```
sudo usermod -a -G render,video $LOGNAME
```
Reboot the system:
```
sudo reboot
```

See Setting Permissions for Groups for more information.

Post-install verification checks#

Run these post-installation checks to verify that the installation is complete:

Verify that the current user is added to the render and video groups:
```
groups
```
Expected result:
```
<username> adm cdrom sudo dip video plugdev render lpadmin lxd sambashare
```
<username> indicates the current user, and this result will vary in your environment.

Check if the GPU is listed as an agent:

rocminfo

Expected result:

*******
Agent 2
*******
  Name:                    gfx1151
  Uuid:                    GPU-XX
  Marketing Name:          Radeon 8050S Graphics
  Vendor Name:             AMD
  [...]

See Installing the all open use case for additional troubleshooting tips.

Configure shared memory#

ROCm utilizes a shared system memory pool, and is configured by default to half the system memory.

Note:
AMD recommends setting the minimum dedicated VRAM in the BIOS (0.5GB), and setting the TTM limit to a larger amount.

This amount can be increased by changing the kernel’s Translation Table Manager (TTM) page setting, available at /sys/module/ttm/parameters/pages_limit. The units are in pages, and must be converted. A helper script is available to assist with configuration.

Install the pipx utility.
```
sudo apt install pipx
```
Add the path for pipx installed wheels into the system search path.
```
pipx ensurepath
```
Install the amd-debug-tools wheel from PyPi.
```
pipx install amd-debug-tools
```
Run the amd-ttm tool to query the current settings for shared memory.
```
amd-ttm
```
Reconfigure shared memory settings by using the --set argument (units in GB).
```
amd-ttm --set <NUM>
```
Reboot the system for changes to take effect.

`amd-ttm` usage examples#

Query effective memory settings in the current kernel

amd-ttm
💻 Current TTM pages limit: 16469033 pages (62.82 GB)
💻 Total system memory: 125.65 GB

Set usable shared memory

❯ amd-ttm --set 100
🐧 Successfully set TTM pages limit to 26214400 pages (100.00 GB)
🐧 Configuration written to /etc/modprobe.d/ttm.conf
○ NOTE: You need to reboot for changes to take effect.
Would you like to reboot the system now? (y/n): y

Clear TTM setting and revert to kernel defaults

❯ amd-ttm --clear
🐧 Configuration /etc/modprobe.d/ttm.conf removed
Would you like to reboot the system now? (y/n): y

Advanced install methods#

For advanced install methods, such as Multi-Version and Package Manager, refer to Install Package Manager for Ryzen APUs and AMD GPU Install Script.

Uninstall ROCm#

Run the following command to uninstall the ROCm software stack and other Ryzen software for Linux components:

sudo amdgpu-uninstall

Upgrade to newer Ryzen Software versions for Linux#

The recommended method to upgrade is to uninstall, followed by an install.

Note
Ryzen Software for Linux does not support in-place upgrades.

Install Package Manager#

Caution
Ensure that the Installation prerequisites are met before installing.

Note
The following installation steps also apply when upgrading from a previous ROCm version.
Package Manager install is an advanced installation method that provides the user with extra flexibility and robustness.

Registering ROCm repositories#

Package signing key#

Download and convert the package signing key.

# Make the directory if it doesn't exist yet.
# This location is recommended by the distribution maintainers.
sudo mkdir --parents --mode=0755 /etc/apt/keyrings
# Download the key, convert the signing-key to a full
# keyring required by apt and store in the keyring directory
wget https://repo.radeon.com/rocm/rocm.gpg.key -O - | \
    gpg --dearmor | sudo tee /etc/apt/keyrings/rocm.gpg > /dev/null

Note
The GPG key may change; ensure it is updated when installing a new release. If the key signature verification fails while updating, re-add the key from the ROCm to the apt repository as mentioned above.

Register packages#

Ubuntu 24.04

sudo tee /etc/apt/sources.list.d/rocm.list << EOF
deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/7.2/ noble main
EOF
sudo tee /etc/apt/preferences.d/rocm-pin-600 << EOF
Package: *
Pin: release o=repo.radeon.com
Pin-Priority: 600
EOF
sudo apt update

Installation#

Prepare the system#

For ROCm on Ryzen, it is required to operate on the 6.14-1018 OEM kernel or newer.

To install the kernel, please run the following command:

sudo apt update && sudo apt install linux-oem-24.04c

Once installation is complete, please reboot your system and ensure that you’ve booted into the correct kernel:
```
uname -r
```
Note
This returns a 6.14-1018 or newer based string.
Ensure that the system is up to date:
```
sudo apt upgrade -y
```

Install ROCm#

sudo apt install rocm

ROCm installation can be tailored to your requirements using one more combinations of ROCm meta packages:

To use pre-built ROCm libraries and tools, include ROCm runtime packages in the installation step.
To develop and build individual ROCm libraries and tools, include ROCm developer packages in the installation step.

ROCm runtime packages#

Meta package	Description	Legacy use case ¹
rocm	All ROCm core packages, tools, and libraries.	rocm
rocm-hip-libraries	HIP libraries optimized for the AMD platform.	Legacy use case does not exist.
rocm-hip-runtime	Run HIP applications written for the AMD platform.	hip
rocm-language-runtime	ROCm runtime environment for running applications on the AMD platform.	lrt
rocm-ml-libraries	Key machine learning libraries. Includes MIOpen.	mllib
rocm-opencl-runtime	Run OpenCL-based applications on the AMD platform.	opencl

Other packages#

Package	Description	Legacy use case ¹
amdgpu-lib	For users of graphics applications which require the open source Mesa 3D graphics and multimedia libraries. This package is primarily used for Radeon GPUs.	graphics

ROCm developer packages#

Meta package	Description	Legacy use case ¹
rocm-developer-tools	Debug and profile HIP applications.	rocmdevtools
rocm-hip-runtime-devel	Develop applications on HIP or port from CUDA.	Legacy use case does not exist.
rocm-hip-sdk	Develop or port HIP applications and libraries for the AMD platform.	hiplibsdk
rocm-ml-sdk	Develop and run machine learning applications for AMD.	mlsdk
rocm-opencl-sdk	Develop OpenCL-based applications for the AMD platform.	openclsdk
rocm-openmp-sdk	Develop OpenMP-based applications for the AMD software.	openmpsdk

Footnote#

[1] (1,2)
Starting from ROCm 6.4.2, “Legacy use cases” in amdgpu-install are replaced by the equivalent meta package. In addition, the following amdgpu-install use cases: asan, rocmdev, multimedia, multimediasdk, amf, and workstation are deprecated.

Post-installation#

Complete the Post-installation instructions.

Uninstalling#

Uninstall ROCm meta packages#

sudo apt autoremove rocm
sudo apt autoremove rocm-core

Remove ROCm repositories#

# Remove the repositories
sudo rm /etc/apt/sources.list.d/rocm.list
# Clear the cache and clean the system
sudo rm -rf /var/cache/apt/*
sudo apt clean all
sudo apt update

Important
To apply all settings, reboot your system.

Install PyTorch for ROCm#

Preparing Your System#

For PyTorch on Ryzen, it is required to operate on the 6.14-1018 OEM kernel or newer.

To install the kernel, please run the following command:

sudo apt update && sudo apt install linux-oem-24.04c

Once installation is complete, please reboot your system and ensure that you’ve booted into the correct kernel:
```
uname -r
```
Note
This returns a 6.14-1018 or newer based string.
If the kernel is correct, ensure the system is up to date:
```
sudo apt update && sudo apt upgrade -y
```

Install PyTorch via PIP Installation#

AMD recommends the PIP install method to create a PyTorch environment when working with ROCm™ for machine learning development.

Check Pytorch.org for latest PIP install instructions and availability. See Compatibility matrices for support information.

** PCIe Atomics ROCm is an extension of HSA platform architecture and shares queuing model, memory model, signaling, and synchronization protocols.

Platform atomics are integral to perform queuing and signaling memory operations, where there may be multiple writers across CPU and GPU agents.

For more details, see How ROCm uses PCIe atomics.

Enter the following command to unpack and begin setup:
```
sudo apt install python3-pip -y
```
Enter this command to update the pip wheel:
```
pip3 install --upgrade pip wheel
```

Enter the commands to install Torch and Torchvision for ROCm AMD GPU support. This may take several minutes.

Important AMD recommends proceeding with ROCm WHLs available on repo.radeon. The ROCm WHLs available at PyTorch Foundation are not tested extensively by AMD as the WHLs change regularly when the nightly builds are updated. When manually downloading WHLs from repo.ryzen, ensure to select the compatible WHLs for specific Python versions. See Compatibility matrices for support information.

Ubuntu 24.04

wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torch-2.9.1%2Brocm7.2.0.lw.git7e1940d4-cp312-cp312-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torchvision-0.24.0%2Brocm7.2.0.gitb919bd0c-cp312-cp312-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/triton-3.5.1%2Brocm7.2.0.gita272dfa8-cp312-cp312-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torchaudio-2.9.0%2Brocm7.2.0.gite3c6ee2b-cp312-cp312-linux_x86_64.whl
pip3 uninstall torch torchvision triton torchaudio
pip3 install torch-2.9.1+rocm7.2.0.lw.git7e1940d4-cp312-cp312-linux_x86_64.whl torchvision-0.24.0+rocm7.2.0.gitb919bd0c-cp312-cp312-linux_x86_64.whl torchaudio-2.9.0+rocm7.2.0.gite3c6ee2b-cp312-cp312-linux_x86_64.whl triton-3.5.1+rocm7.2.0.gita272dfa8-cp312-cp312-linux_x86_64.whl

Note: The --break-system-packages flag must be added when installing wheels for Python 3.12 in a non-virtual environment.

Note: To enable AOTriton kernels, run the following command before running any PyTorch scripts:
export TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1

Next, verify the PyTorch installation.

Verify Your PyTorch Installation#

Confirm if PyTorch is correctly installed.

Verify if PyTorch is installed and detecting the GPU compute device:

python3 -c 'import torch' 2> /dev/null && echo 'Success' || echo 'Failure'

Expected result:

Success

Enter command to check if the GPU is accessible from PyTorch. In the PyTorch framework, torch.cuda is a generic way to access the GPU. This will only access an AMD GPU if one is available.
```
python3 -c 'import torch; print(torch.cuda.is_available())'
```
Expected result:
```
True
```

Enter the command to display the installed GPU device name:

python3 -c "import torch; print(f'device name [0]:', torch.cuda.get_device_name(0))"

Expected result: Example: device name [0]: AMD Radeon Graphics

device name [0]: <Supported AMD GPU>

Enter the command to display component information within the current PyTorch environment:

python3 -m torch.utils.collect_env

Expected result:

PyTorch version
ROCM used to build PyTorch
OS
Is CUDA available
GPU model and configuration
HIP runtime version
MIOpen runtime version

The environment set-up is complete, and the system is ready for use with PyTorch to work with machine learning models and algorithms.

How to guide - Use ROCm on Ryzen#

This guide walks you through the various installation processes required to pair ROCm™ with the latest high-end AMD Ryzen™ APUs, and get started on a fully-functional environment for AI and ML development.

PyTorch via PIP installation#

AMD recommends the PIP install method to create a PyTorch environment when working with ROCm™ for machine learning development.

Note
To install the following wheels, Python 3.12 must be installed.

Prerequisites#

For the 7.2 PyTorch on Windows release, the 26.1.1 graphics driver must be installed.

Install PyTorch via PIP#

Enter the commands to set up ROCm environment.

CMD Prompt

pip install --no-cache-dir ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_core-7.2.0.dev0-py3-none-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_devel-7.2.0.dev0-py3-none-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_libraries_custom-7.2.0.dev0-py3-none-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm-7.2.0.dev0.tar.gz

PowerShell

pip install --no-cache-dir `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_core-7.2.0.dev0-py3-none-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_devel-7.2.0.dev0-py3-none-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_libraries_custom-7.2.0.dev0-py3-none-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm-7.2.0.dev0.tar.gz

Enter the commands to install torch, torchvision and torchaudio for ROCm AMD GPU support.

Note
This may take several minutes. See Compatibility matrices for support information.

CMD Prompt

pip install --no-cache-dir ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torch-2.9.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torchaudio-2.9.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torchvision-0.24.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl

PowerShell

pip install --no-cache-dir `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torch-2.9.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torchaudio-2.9.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torchvision-0.24.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl

Verify PyTorch installation#

Confirm if PyTorch is correctly installed.

Verify if Pytorch is installed and detecting the GPU compute device.
```
python -c "import torch" 2>nul && echo Success || echo Failure
```
Expected result:
```
Success
```

Enter command to test if the GPU is available.

python -c "import torch; print(torch.cuda.is_available())"

Expected result:

True

Enter command to display installed GPU device name.

python -c "import torch; print(f'device name [0]:', torch.cuda.get_device_name(0))"

Example result: device name [0]: AMD Radeon Graphics

device name [0]: <Supported AMD GPU>

Enter command to display component information within the current PyTorch environment.

python -m torch.utils.collect_env

Example result:

PyTorch version: 2.9.1+rocmsdk20260116
Is debug build: False
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: 7.2.26024-f6f897bd3d

OS: Microsoft Windows 11 Pro (10.0.26100 64-bit)
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.12.10 (tags/v3.12.10:0cc8128, Apr  8 2025, 12:21:36) [MSC v.1943 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-11-10.0.26100-SP0
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to:
GPU models and configuration: AMD Radeon(TM) 8060S Graphics (gfx1151)
Nvidia driver version: Could not collect
cuDNN version: Could not collect
Is XPU available: False
HIP runtime version: 7.2.26024
MIOpen runtime version: 3.5.1
Is XNNPACK available: True

Environment set-up is complete, and the system is ready for use with PyTorch to work with machine learning models, and algorithms.

Install HIP SDK for Radeon GPUs#

The HIP SDK for Windows brings a subset of the ROCm platform to Windows. It provides APIs and tooling to leverage the computational power of accelerators and GPUs to create high-performance, portable applications using HIP.

For HIP SDK installation instructions, component support, system requirements and limitations, refer to HIP SDK installation for Windows.

Radeon Usecases#

Refer to the applicable guides to optimize Radeon specific usecase performance.

Radeon Usecases#

Refer to the applicable guides to optimize Radeon specific usecase performance.

LLM#

Refer to the applicable guides to optimize LLM usecase performance.

Note

Radeon GPUs support vLLM usecases on Linux OS.

vLLM Docker image#

Virtual Large Language Model (vLLM) is a fast and easy-to-use library for LLM inference and serving.

Model support is enabled via a vLLM Docker image that must be built separately (in addition to ROCm) for the current release.

For additional information, visit the AMD vLLM GitHub page.

Note that this is a benchmarking demo/example. Installation for other vLLM models/configurations may differ.

Prerequisites#

GitHub is authenticated.

Additional information#

AMD recommends 40GB GPU for 70B usecases.
Ensure that your GPU has enough VRAM for the chosen model.
This example highlights use of the AMD vLLM Docker using Llama-3 70B with GPTQ quantization (as shown at Computex).
However, performance is not limited to this specific Hugging Face model, and other vLLM supported models can also be used.

Download and install Docker image#

Download Docker image#

Select the applicable Ubuntu version to download the compatible Docker image before starting.

Ubuntu 24.04/Python 3.12/PyTorch 2.9

docker pull rocm/vllm-dev:rocm7.2_navi_ubuntu24.04_py3.12_pytorch_2.9_vllm_0.14.0rc0

Ubuntu 22.04/Python 3.10/PyTorch 2.9

docker pull rocm/vllm-dev:rocm7.2_navi_ubuntu22.04_py3.10_pytorch_2.9_vllm_0.14.0rc0

Note

For more information, see rocm/vllm-dev.

Installation#

Follow these steps to build a vLLM Docker image.

Start the Docker container.

Important!
Refer to WSL specific configurations for instructions when working in a WSL environment.

docker run -it \
  --privileged \
  --device=/dev/kfd \
  --device=/dev/dri \
  --network=host \
  --group-add sudo \
  -w /app/vllm/ \
  --name <container_name> \
<image_name> \
  /bin/bash

Note

The container_name is user defined. Ensure to name your Docker using this value.

WSL-specific configurations

Optional: Only applicable when using a WSL configuration
Select the applicable vLLM instructions, based on your specific WSL configuration.

Ubuntu 24.04/Python 3.12/PyTorch 2.9

docker run -it \
--network=host \
--group-add=video \
--ipc=host \
--cap-add=SYS_PTRACE \
--security-opt seccomp=unconfined \
--device /dev/dxg \
--entrypoint /bin/bash \
-v /usr/lib/wsl/lib/libdxcore.so:/usr/lib/libdxcore.so \
-v /opt/rocm/lib/libhsa-runtime64.so.1:/opt/rocm/lib/libhsa-runtime64.so.1 \
-w /app/vllm/ \
--name vllm_rocm_container \
rocm/vllm-dev:rocm7.2_navi_ubuntu24.04_py3.12_pytorch_2.9_vllm_0.14.0rc0
-c "sed -i 's/is_rocm = False/is_rocm = True/g' /opt/conda/envs/py_3.12/lib/python3.12/site-packages/vllm/platforms/__init__.py && /bin/bash"

Ubuntu 22.04/Python 3.10/PyTorch 2.9

docker run -it \
--network=host \
--group-add=video \
--ipc=host \
--cap-add=SYS_PTRACE \
--security-opt seccomp=unconfined \
--device /dev/dxg \
--entrypoint /bin/bash \
-v /usr/lib/wsl/lib/libdxcore.so:/usr/lib/libdxcore.so \
-v /opt/rocm/lib/libhsa-runtime64.so.1:/opt/rocm/lib/libhsa-runtime64.so.1 \
-w /app/vllm/ \
--name vllm_rocm_container \
rocm/vllm-dev:rocm7.2_navi_ubuntu22.04_py3.10_pytorch_2.9_vllm_0.14.0rc0
-c "sed -i 's/is_rocm = False/is_rocm = True/g' /opt/conda/envs/py_3.10/lib/python3.10/site-packages/vllm/platforms/__init__.py && /bin/bash"

Clone the Hugging Face GitHub repository within the Docker container.

apt update
apt install git-lfs
git lfs clone https://huggingface.co/TechxGenus/Meta-Llama-3-70B-Instruct-GPTQ

Run benchmarks with the Docker container.

vllm bench latency --model /app/vllm/Meta-Llama-3-70B-Instruct-GPTQ -q gptq --batch-size 1 --input-len 1024 --output-len 1024 --max-model-len 2048

This is a vllm CLI command for the latency mode. Similar parameters can be used for bench or serve modes, but they are separate modes and use different subcommands.

This can also be called using python -m vllm.entrypoints.cli.main bench latency...

For additional information, refer to vLLM CLI Guide - vLLM

Note
Ensure that the model is downloaded and vLLM checkout is set to your current directory within the container described in Step 3.

Note
Select the preferred environment variable prior to running models using vLLM in V1 mode:

Selection

Prefill

Decode

Flags

TRITON_ATTN (default)

kernel_unified_attention

same unified

None
or --attention-config.backend TRITON_ATTN

ROCM_ATTN (custom paged attention)

context_attention_fwd

paged_attention_rocm

VLLM_V1_USE_PREFILL_DECODE_ATTENTION=1
or --attention-config.backend ROCM_ATTN --attention-config.use_prefill_decode_attention=true

ROCM_AITER_UNIFIED_ATTN

AITER unified_attention

same unified

VLLM_ATTENTION_BACKEND=ROCM_AITER_UNIFIED_ATTN
or --attention-config.backend ROCM_AITER_UNIFIED_ATTN

GEMM tuning for model inferencing with vLLM#

Note
Tuning must be done for specific tensor-parallel-size.

Collect GEMM shape details#

Collect GEMM shape details used in this model inference, and ensure tp is specified based on targeted setup accordingly (split over N GPUs).

VLLM_TUNE_GEMM=1 VLLM_UNTUNE_FILE=untuned_gemm.csv python <vllm_path>/benchmarks/benchmark_throughput.py --model <model_path> --trust-remote-code --dataset <dataset_path>/ShareGPT_V3_unfiltered_cleaned_split.json --num_prompts 1000 [--distributed_executor_backend mp --tensor-parallel-size N]

Conduct GEMM tuning#

The indtype/outdtype must be correctly specified, and aligned with dtype used for model inferencing.

For quantized int4 models, use f16 or bf16. Refer to ROCm vLLM Github (Sep 23, 2024, included in v0.6.1.post1+rocm release) as a starting point.

The generated untuned_gemm.csv contains dtype info that is used by default, and is only overridden by indtype and outdtype when specified.

Important
Do not use the tp option, as it is designed for other usecases.

Note
If gradlib fails with OOM then set CACHE_INVALIDATE_BUFFERS to a primary number (such as 11 or 7 or 3 or even 1).

python <vllm_path>/gradlib/gradlib/gemm_tuner.py --input_file untuned_gemm.csv --tuned_file tuned_gemm_tpN.csv [--indtype f16 --outdtype f16]

Run vLLM inference with tuned GEMM#

Enter the following command.

VLLM_TUNE_FILE=tuned_gemm_tpN.csv python <vllm_path>/benchmarks/benchmark_throughput.py --model <model_path> --trust-remote-code --dataset <dataset_path>/ShareGPT_V3_unfiltered_cleaned_split.json --num_prompts 1000 [--distributed_executor_backend mp --tensor-parallel-size N]

LLM inference with PyTorch + Huggingface transformers#

Install Huggingface transformers#

Follow these steps to install Huggingface transformers.

Prerequisites#

ROCm is installed. For instructions, refer to Install Radeon Software for Linux with ROCm.

Installation#

Follow these steps to install transformers.

Install the Python venv package for the applicable Python version.
Python 3.12/Ubuntu 24.04
sudo apt install python3.12-venv
Python 3.10/Ubuntu 22.04
sudo apt install python3.10-venv

Create a Python virtual environment.

python3 -m venv llm-venv
source llm-venv/bin/activate

Install the latest PyTorch ROCm wheels in the environment created.
Note
Refer to Install Pytorch for Radeon GPUs for more comprehensive install instructions. Proceed to install within the environment if the wheels are already downloaded to the system. Example command:
pip3 install <torch wheel> <torchaudio wheel> <torchvision wheel> <triton wheel>

Install transformers and required packages.

pip install transformers
pip install accelerate

(Optional) Install HuggingFaceHub, which is the Python client to download, and upload models to Hugging Face.
```
pip install huggingface-hub
hf auth login
```

LLM inference#

import torch
from transformers import pipeline
model_id = "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B"
pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)
messages = [
    {"role": "system", "content": "You are a helpful technology enthusiast."},
    {"role": "user", "content": "What is AMD Radeon?"},
]
outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])

Model support matrix#

Model	Link	Supported
Llama-3.2-1B-Instruct	https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct	Yes
Llama-3.2-3B-Instruct	https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct	Yes
DeepSeek-R1-Distill-Qwen-1.5B	https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B	Yes

Note
Proprietary Meta access is required for Llama models.
Alternatively, open-source versions can be found here:

unsloth/Llama-3.2-1B · Hugging Face

unsloth/Llama-3.2-3B · Hugging Face

Llama.cpp pre-built binaries#

llama.cpp is an open-source framework for Large Language Model (LLM) inference that runs on both central processing units (CPUs) and graphics processing units (GPUs).

This document provides installation instructions for the AMD-validated llama.cpp prebuilt binaries. These are pre-compiled, stable executables (like server and llama-bench) that are ready to run on a Linux system without requiring any compilation.

Download the AMD-validated Linux binary package and extract it.

Ubuntu 24.04

Download the prebuilt binary package.

wget -O llama-bin-linux.zip "https://repo.radeon.com/rocm/llama.cpp/linux/rocm-rel-7.2/llama-b7782-ubuntu-24.04-rocm-7.2.0-gfx110X-gfx115X-gfx120X-x64.zip"

Unzip the package into a new directory.

unzip llama-bin-linux.zip -d ./llama_cpp_binaries

Navigate into the inner directory.

cd ./llama_cpp_binaries/<specific_folder_name>

Make binaries executable. Once in the new directory, grant the llama-server, llama-bench and llama-cli tool execute permissions.
```
chmod +x ./llama-server
chmod +x ./llama-bench
chmod +x ./llama-cli
```
Download a Test Model. These binaries are the “engine”; you still need a model file (in GGUF format) to run. For this tutorial, we will download the GPT-OSS-20B for testing.
Ubuntu 24.04
wget -O test_model.gguf "https://huggingface.co/ggml-org/gpt-oss-20b-GGUF/resolve/main/gpt-oss-20b-mxfp4.gguf"

Run Llama-Server. llama-server is a lightweight, OpenAI-compatible web server included with llama.cpp that hosts your model locally. Once running, it provides a simple web interface that allows you to chat with the model directly in your browser.

Ubuntu 24.04

# Start the server
# -ngl 99: Offload all layers to your AMD GPU (Crucial for performance)
# -c: Context Length
# -fa: Enable Flash Attention to reduce memory usage and increase speed
./llama-server -m test_model.gguf -c 2048 -ngl 99 -fa on --port 8080

(Optional) Run a benchmark. Now, run the llama-bench tool against the test model. This command will load the model and run a standardized performance test, measuring your system’s prompt processing (PP) and token generation (TG) speed.
Ubuntu 24.04
# Run the benchmark with the downloaded model. # -m: specifies the model file ./llama-bench -m ./test_model.gguf -fa 1

ComfyUI#

Install ComfyUI and its extensions on your system.

Install ComfyUI and MIGraphX extension#

Install ComfyUI and the MIGraphX extension on AMD GPUs.

Prerequisites#

ROCm is installed.

Installation#

Follow these steps to install ComfyUI.

Install the Python venv package for the applicable Python version.
Python 3.12/Ubuntu 24.04
sudo apt install python3.12-venv
Python 3.10/Ubuntu 22.04
sudo apt install python3.10-venv

Create a Python virtual environment.

python3 -m venv comfyui-venv
source comfyui-venv/bin/activate

Install the latest PyTorch ROCm wheels in the environment created.
Note
Refer to Install Pytorch for Radeon GPUs for more comprehensive install instructions. Proceed to install within the environment if the wheels are already downloaded to the system. Example command:
pip3 install <torch wheel> <torchaudio wheel> <torchvision wheel> <triton wheel>

Clone the ComfyUI repository.

git clone https://github.com/comfyanonymous/ComfyUI.git && cd ComfyUI

Note

See ComfyUI Github for more information.

Install ComfyUI requirements.
```
pip install -r requirements.txt
```
Launch ComfyUI.

Note
Users must return to the home directory of the repository in order to run main.py.
```
python3 main.py
```
Click to URL to open ComfyUI in a browser.

Note
After running the above command, ensure that the Pytorch version seen in the terminal is the ROCm 7.2 specific one.

Note
If running on low-memory configs, try adding the --lowvram and --disable-pinned-memory parameters to the run command.
From the ComfyUI frontend, click on Templates → Search for ‘SDXL Turbo’ → Follow prompt for model install and place it into the ‘checkpoints’ folder under ComfyUI>models>checkpoints.
From the ComfyUI frontend, click ‘Run’ to begin image generation.

Installing the MIGraphX node for ComfyUI#

The MIGraphX Node for ComfyUI offers accelerated inference on Stable Diffusion 3 models.

Note

Refer to ComfyUI MIGraphX Github for more information.

Install the MIGraphX node.

cd ComfyUI/custom_nodes
git clone https://github.com/pnikolic-amd/ComfyUI_MIGraphX.git
cd ComfyUI_MIGraphX
pip install -r requirements.txt

Launch ComfyUI.

Note

The ROCm path to Python must be exported to prevent a module missing error for MIGraphX.
export PYTHONPATH=/opt/rocm-X.X.X/lib:$PYTHONPATH
Replace the ROCm version with your applicable version. For example, /opt/rocm-6.4.1/lib.

cd ComfyUI
# For the best performance, run the following environment variable:
export MIGRAPHX_MLIR_USE_SPECIFIC_OPS="attention"
python3 main.py

Using the MIGraphX node for ComfyUI#

The MIGraphX Node for ComfyUI includes ready-to-use templates, each preconfigured with optimized settings for the supported Stable Diffusion models.

To access them, go to: Workflow > Browse Templates > ComfyUI_MIGraphX
Each template is optimized for a specific Stable Diffusion model. Be sure to load the corresponding model before using the template.

Radeon Usecases#

Refer to the applicable guides to optimize Radeon specific usecase performance.

LLM#

Refer to the applicable guides to optimize LLM/vLLM usecase performance.

Note

Radeon GPUs do not support vLLM usecases on Windows OS.

LLM inference with PyTorch + Huggingface transformers#

Install Huggingface transformers#

Follow these steps to install Huggingface transformers.

Prerequisites#

Python 3.12 is installed.
26.1.1 graphics driver is installed. Refer to Install Pytorch for Radeon GPUs for more information.

Installation#

Follow these steps to install Transformers with Powershell.

Create and activate a Python virtual environment in a directory of your choice.
```
python -m venv llm-venv
llm-venv\Scripts\activate
```

Enter the commands to set up ROCm environment.

CMD Prompt

pip install --no-cache-dir ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_core-7.2.0.dev0-py3-none-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_devel-7.2.0.dev0-py3-none-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_libraries_custom-7.2.0.dev0-py3-none-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm-7.2.0.dev0.tar.gz

PowerShell

pip install --no-cache-dir `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_core-7.2.0.dev0-py3-none-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_devel-7.2.0.dev0-py3-none-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_libraries_custom-7.2.0.dev0-py3-none-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm-7.2.0.dev0.tar.gz

Enter the commands to install torch, torchvision and torchaudio for ROCm AMD GPU support.

Note
This may take several minutes. See Compatibility matrices for support information.

CMD Prompt

pip install --no-cache-dir ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torch-2.9.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torchaudio-2.9.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torchvision-0.24.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl

PowerShell

pip install --no-cache-dir `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torch-2.9.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torchaudio-2.9.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torchvision-0.24.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl

Install transformers.

Install a specific release version with the following command:
```
pip install transformers
```
(Optional) Install HuggingFaceHub, which is the Python client to download, and upload models to Hugging Face.
```
pip install huggingface-hub
hf auth login # login if desired
```

LLM inference#

Python
import torch
from transformers import pipeline
model_id = "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B"
pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)
messages = [
    {"role": "system", "content": "You are a helpful technology enthusiast."},
    {"role": "user", "content": "What is AMD Radeon?"},
]
outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])

Model support matrix#

Model	Link	Supported
Llama-3.2-1B-Instruct	https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct	Yes
Llama-3.2-3B-Instruct	https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct	Yes
DeepSeek-R1-Distill-Qwen-1.5B	https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B	Yes

Note
Proprietary Meta access is required.
Alternatively, open-source versions can be found here:

unsloth/Llama-3.2-1B · Hugging Face

unsloth/Llama-3.2-3B · Hugging Face

Llama.cpp pre-built binaries#

llama.cpp is an open-source framework for Large Language Model (LLM) inference that runs on both central processing units (CPUs) and graphics processing units (GPUs).

This document provides installation instructions for the AMD-validated llama.cpp prebuilt binaries. These are pre-compiled, stable executables (like server and llama-bench) that are ready to run on a Linux system without requiring any compilation.

Download the AMD-validated Windows binary package and extract it.

Windows

Download the prebuilt binary package.

curl.exe -o llama-bin-windows.zip "https://repo.radeon.com/rocm/llama.cpp/windows/rocm-rel-7.2/llama-b7782-windows-rocm-7.2.0-gfx110X-gfx115X-gfx120X-x64.zip"

Unzip the package into a new directory. (Powershell)

Expand-Archive -Path "llama-bin-windows.zip" -DestinationPath ".\llama_cpp_binaries"

Navigate into the inner directory

cd ./llama_cpp_binaries/<specific_folder_name>

Download a Test Model. These binaries are the “engine”; you still need a model file (in GGUF format) to run. For this tutorial, we will download the GPT-OSS-20B for testing.
Ubuntu 24.04
curl.exe -L -o test_model.gguf "https://huggingface.co/ggml-org/gpt-oss-20b-GGUF/resolve/main/gpt-oss-20b-mxfp4.gguf"

Run Llama-Server. llama-server is a lightweight, OpenAI-compatible web server included with llama.cpp that hosts your model locally. Once running, it provides a simple web interface that allows you to chat with the model directly in your browser.

Ubuntu 24.04

# Start the server
# -ngl 99: Offload all layers to your AMD GPU (Crucial for performance)
# -c: Context Length
# -fa: Enable Flash Attention to reduce memory usage and increase speed
.\llama-server.exe -m test_model.gguf -c 2048 -ngl 99 -fa on --port 8080

(Optional) Run a benchmark. Now, run the llama-bench tool against the test model. This command will load the model and run a standardized performance test, measuring your system’s prompt processing (PP) and token generation (TG) speed.
Ubuntu 24.04
# Run the benchmark with the downloaded model. # -m: specifies the model file .\llama-bench.exe -m .\test_model.gguf -fa 1

ComfyUI#

Install ComfyUI on your system.

Install ComfyUI#

Install ComfyUI on AMD GPUs.

Installation#

Follow these steps to install ComfyUI.

Install PyTorch (ROCm) wheels. For instructions, see Install PyTorch for Radeon GPUs
Clone the ComfyUI repository.
```
git clone https://github.com/comfyanonymous/ComfyUI.git
```
Note
See ComfyUI Github for more information.
Install ComfyUI requirements.
```
pip install -r requirements.txt
```
Launch ComfyUI.

Note
Users must return to the home directory of the repository in order to run main.py.
```
python main.py
```
Click to URL to open ComfyUI in a browser.

Note
After running the above command, ensure that the Pytorch version seen in the terminal is the ROCm 7.2 specific one.

Note
If running on low-memory configs, try adding the --lowvram and --disable-pinned-memory parameters to the run command.
From the ComfyUI frontend, click on Templates → Search for ‘SDXL Turbo’ → Follow prompt for model install and place it into the ‘checkpoints’ folder under ComfyUI>models>checkpoints.
From the ComfyUI frontend, click ‘Run’ to begin image generation.

Ryzen Usecases#

Refer to the applicable guides to optimize Ryzen specific usecase performance.

Ryzen Usecases#

Refer to the applicable guides to optimize Ryzen specific usecase performance.

LLM#

Refer to the applicable guides to optimize LLM usecase performance.

vLLM Linux Docker Image#

Virtual Large Language Model (vLLM) is a fast and easy-to-use library for LLM inference and serving, providing greater optimizations and performance.

For additional information, visit the AMD vLLM GitHub page.

Note
This is a benchmarking demo/example. Installation for other vLLM models/configurations may differ.

Additional information

Ensure Docker is installed on your system. Refer to this link for more information.

This docker image supports gfx1151 and gfx1150.

This example highlights use of the AMD vLLM Docker using deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B. Other models vLLM supported models can be used too.

Download and install Docker image#

Download Docker image#

Select the applicable Ubuntu version to download the compatible Docker image before starting.

Ubuntu 24.04/Python 3.12/PyTorch 2.9

docker pull rocm/vllm-dev:rocm7.2_navi_ubuntu24.04_py3.12_pytorch_2.9_vllm_0.14.0rc0

Note
For more information, see rocm/vllm-dev.

Installation#

Follow these steps to build a vLLM Docker image and benchmark a model.

Start the Docker container.

docker run -it \
  --privileged \
  --device=/dev/kfd \
  --device=/dev/dri \
  --network=host \
  --group-add sudo \
  -w /app/vllm/ \
  --name <container_name> \
<image_name> \
  /bin/bash

Note
You can find the <image_name> by running docker images. The container_name is user defined. Ensure to name your Docker using this value.

Run benchmarks with the Docker container.
```
vllm bench latency --model /app/vllm/Meta-Llama-3-70B-Instruct-GPTQ -q gptq --batch-size 1 --input-len 1024 --output-len 1024 --max-model-len 2048
```
Note
This is a vllm CLI command for the latency mode. Similar parameters can be used for bench or serve modes, but they are separate modes and use different subcommands.

This can also be called using python -m vllm.entrypoints.cli.main bench latency...

For additional information, refer to vLLM CLI Guide - vLLM

Additional Usage#

vLLM is optimized to serve LLMs faster and more efficiently, especially for applications requiring high throughput and scalability. See Quickstart - OpenAI Compatible Server for more information.
To run offline inference, see Quickstart - Offline Batched Inference for more information.

Note
If you experience errors with torch.distributed, running export GLOO_SOCKET_IFNAME=lo may resolve the issue.

LLM inference with PyTorch + Huggingface transformers#

Follow these steps to install Huggingface transformers.

Prerequisites#

ROCm is installed. For instructions, refer to Install Ryzen Software for Linux with ROCm.

Installation#

Follow these steps to install Transformers.

Install the Python venv package for the applicable Python version.
Python 3.12/Ubuntu 24.04
sudo apt install python3.12-venv

Create a Python virtual environment.

python3 -m venv llm-venv
source llm-venv/bin/activate

Install the latest PyTorch ROCm wheels in the environment created.
Note
Refer to Install Pytorch for Ryzen APUs for more comprehensive install instructions. Proceed to install within the environment if the wheels are already downloaded to the system. Example command:
pip3 install <torch wheel> <torchaudio wheel> <torchvision wheel> <triton wheel>

Install transformers and required packages.

pip install transformers
pip install accelerate

(Optional) Install HuggingFaceHub, which is the Python client to download, and upload models to Hugging Face.
```
pip install huggingface-hub
hf auth login
```

LLM inference#

import torch
from transformers import pipeline
model_id = "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B"
pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)
messages = [
    {"role": "system", "content": "You are a helpful technology enthusiast."},
    {"role": "user", "content": "What is AMD Radeon?"},
]
outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])

Model support matrix#

Model	Link	Supported
Llama-3.2-1B-Instruct	https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct	Yes
Llama-3.2-3B-Instruct	https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct	Yes
DeepSeek-R1-Distill-Qwen-1.5B	https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B	Yes

Note
Proprietary Meta access is required for Llama models.
Alternatively, open-source versions can be found here:

unsloth/Llama-3.2-1B · Hugging Face

unsloth/Llama-3.2-3B · Hugging Face

Llama.cpp pre-built binaries#

llama.cpp is an open-source framework for Large Language Model (LLM) inference that runs on both central processing units (CPUs) and graphics processing units (GPUs).

This document provides installation instructions for the AMD-validated llama.cpp prebuilt binaries. These are pre-compiled, stable executables (like server and llama-bench) that are ready to run on a Linux system without requiring any compilation.

Download the AMD-validated Linux binary package and extract it.

Ubuntu 24.04

Download the prebuilt binary package.

wget -O llama-bin-linux.zip "https://repo.radeon.com/rocm/llama.cpp/linux/rocm-rel-7.2/llama-b7782-ubuntu-24.04-rocm-7.2.0-gfx110X-gfx115X-gfx120X-x64.zip"

Unzip the package into a new directory.

unzip llama-bin-linux.zip -d ./llama_cpp_binaries

Navigate into the inner directory.

cd ./llama_cpp_binaries/<specific_folder_name>

Make binaries executable. Once in the new directory, grant the llama-server, llama-bench and llama-cli tool execute permissions.
```
chmod +x ./llama-server
chmod +x ./llama-bench
chmod +x ./llama-cli
```
Download a Test Model. These binaries are the “engine”; you still need a model file (in GGUF format) to run. For this tutorial, we will download the GPT-OSS-20B for testing.
Ubuntu 24.04
wget -O test_model.gguf "https://huggingface.co/ggml-org/gpt-oss-20b-GGUF/resolve/main/gpt-oss-20b-mxfp4.gguf"

Run Llama-Server. llama-server is a lightweight, OpenAI-compatible web server included with llama.cpp that hosts your model locally. Once running, it provides a simple web interface that allows you to chat with the model directly in your browser.

Ubuntu 24.04

# Start the server
# -ngl 99: Offload all layers to your AMD GPU (Crucial for performance)
# -c: Context Length
# -fa: Enable Flash Attention to reduce memory usage and increase speed
./llama-server -m test_model.gguf -c 2048 -ngl 99 -fa on --port 8080

(Optional) Run a benchmark. Now, run the llama-bench tool against the test model. This command will load the model and run a standardized performance test, measuring your system’s prompt processing (PP) and token generation (TG) speed.
Ubuntu 24.04
# Run the benchmark with the downloaded model. # -m: specifies the model file ./llama-bench -m ./test_model.gguf -fa 1

ComfyUI#

Install ComfyUI on your system.

Install ComfyUI#

Install ComfyUI on AMD GPUs.

Prerequisites#

ROCm is installed.

Installation#

Follow these steps to install ComfyUI.

Install the Python venv package for the applicable Python version.
Python 3.12/Ubuntu 24.04
sudo apt install python3.12-venv

Create a Python virtual environment.

python3 -m venv comfyui-venv
source comfyui-venv/bin/activate

Install the latest PyTorch ROCm wheels in the environment created.
Note
Refer to Install Pytorch for Ryzen APUs for more comprehensive install instructions. Proceed to install within the environment if the wheels are already downloaded to the system. Example command:
pip3 install <torch wheel> <torchaudio wheel> <torchvision wheel> <triton wheel>

Clone the ComfyUI repository.

git clone https://github.com/comfyanonymous/ComfyUI.git && cd ComfyUI

Note

See ComfyUI Github for more information.

Install ComfyUI requirements.
```
pip install -r requirements.txt
```
Launch ComfyUI.

Note
Users must return to the home directory of the repository in order to run main.py.
```
python3 main.py
```
Click the URL to open ComfyUI in a browser.

Note
After running the above command, ensure that the Pytorch version seen in the terminal is the ROCm 7.2 specific one.

Note
If running on low-memory configs, try adding the --lowvram and --disable-pinned-memory parameters to the run command.
From the ComfyUI frontend, click on Templates → Search for ‘SDXL Turbo’ → Follow prompt for model install and place it into the ‘checkpoints’ folder under ComfyUI>models>checkpoints.
From the ComfyUI frontend, click ‘Run’ to begin image generation.

Ryzen Usecases#

Refer to the applicable guides to optimize Ryzen specific usecase performance.

LLM#

Refer to the applicable guides to optimize LLM usecase performance.

Note

Ryzen APUs do not support vLLM usecases.

LLM inference with PyTorch + Huggingface transformers#

Install Huggingface transformers#

Follow these steps to install Huggingface transformers.

Prerequisites#

Python 3.12 is installed.
26.1.1 graphics driver is installed. Refer to Install Pytorch for Ryzen APUs for more information.

Installation#

Follow these steps to install Transformers with Powershell.

Create and activate a Python virtual environment in a directory of your choice.
```
python -m venv llm-venv
llm-venv\Scripts\activate
```

Enter the commands to set up ROCm environment.

CMD Prompt

pip install --no-cache-dir ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_core-7.2.0.dev0-py3-none-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_devel-7.2.0.dev0-py3-none-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_libraries_custom-7.2.0.dev0-py3-none-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm-7.2.0.dev0.tar.gz

PowerShell

pip install --no-cache-dir `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_core-7.2.0.dev0-py3-none-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_devel-7.2.0.dev0-py3-none-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm_sdk_libraries_custom-7.2.0.dev0-py3-none-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/rocm-7.2.0.dev0.tar.gz

Enter the commands to install torch, torchvision and torchaudio for ROCm AMD GPU support.

Note
This may take several minutes. See Compatibility matrices for support information.

CMD Prompt

pip install --no-cache-dir ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torch-2.9.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torchaudio-2.9.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl ^
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torchvision-0.24.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl

PowerShell

pip install --no-cache-dir `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torch-2.9.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torchaudio-2.9.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl `
    https://repo.radeon.com/rocm/windows/rocm-rel-7.2/torchvision-0.24.1%2Brocmsdk20260116-cp312-cp312-win_amd64.whl

Install transformers and required packages.

pip install transformers
pip install accelerate   

(Optional) Install HuggingFaceHub, which is the Python client to download, and upload models to Hugging Face.
```
pip install huggingface-hub
hf auth login
```

LLM inference#

import torch
from transformers import pipeline
model_id = "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B"
pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)
messages = [
    {"role": "system", "content": "You are a helpful technology enthusiast."},
    {"role": "user", "content": "What is AMD Radeon?"},
]
outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])

Model support matrix#

Model	Link	Supported
Llama-3.2-1B-Instruct	https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct	Yes
Llama-3.2-3B-Instruct	https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct	Yes
DeepSeek-R1-Distill-Qwen-1.5B	https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B	Yes

Note
Proprietary Meta access is required for Llama models.
Alternatively, open-source versions can be found here:

unsloth/Llama-3.2-1B · Hugging Face

unsloth/Llama-3.2-3B · Hugging Face

Llama.cpp pre-built binaries#

llama.cpp is an open-source framework for Large Language Model (LLM) inference that runs on both central processing units (CPUs) and graphics processing units (GPUs).

This document provides installation instructions for the AMD-validated llama.cpp prebuilt binaries. These are pre-compiled, stable executables (like server and llama-bench) that are ready to run on a Windows system without requiring any compilation.

Download the AMD-validated Windows binary package and extract it.

Windows

Download the prebuilt binary package.

curl.exe -o llama-bin-windows.zip "https://repo.radeon.com/rocm/llama.cpp/windows/rocm-rel-7.2/llama-b7782-windows-rocm-7.2.0-gfx110X-gfx115X-gfx120X-x64.zip"

Unzip the package into a new directory. (Powershell)

Expand-Archive -Path "llama-bin-windows.zip" -DestinationPath ".\llama_cpp_binaries"

Navigate into the inner directory

cd ./llama_cpp_binaries/<specific_folder_name>

Download a Test Model. These binaries are the “engine”; you still need a model file (in GGUF format) to run. For this tutorial, we will download the GPT-OSS-20B for testing.
Windows
curl.exe -L -o test_model.gguf "https://huggingface.co/ggml-org/gpt-oss-20b-GGUF/resolve/main/gpt-oss-20b-mxfp4.gguf"

Run Llama-Server. llama-server is a lightweight, OpenAI-compatible web server included with llama.cpp that hosts your model locally. Once running, it provides a simple web interface that allows you to chat with the model directly in your browser.

Windows

# Start the server
# -ngl 99: Offload all layers to your AMD GPU (Crucial for performance)
# -c: Context Length
# -fa: Enable Flash Attention to reduce memory usage and increase speed
.\llama-server.exe -m test_model.gguf -c 2048 -ngl 99 -fa on --port 8080

(Optional) Run a benchmark. Now, run the llama-bench tool against the test model. This command will load the model and run a standardized performance test, measuring your system’s prompt processing (PP) and token generation (TG) speed.
Windows
# Run the benchmark with the downloaded model. # -m: specifies the model file .\llama-bench.exe -m .\test_model.gguf -fa 1

ComfyUI#

Install ComfyUI on your system.

Install ComfyUI#

Install ComfyUI on AMD GPUs.

Prerequisites#

Ensure Python 3.12 is installed.
Ensure 26.1.1 graphics driver is installed. Refer to Install PyTorch for Ryzen APUs for more information.

Installation#

Follow these steps to install ComfyUI.

Create and activate a Python virtual environment in a directory of your choice.
```
python -m venv comfyui-venv
comfyui-venv\Scripts\activate
```
Install PyTorch (ROCm) wheels. See Install PyTorch for Ryzen APUs for install instructions.
Clone the ComfyUI repository.
```
git clone https://github.com/comfyanonymous/ComfyUI.git
```
Note
See ComfyUI Github for more information.
Install ComfyUI requirements.
```
pip install -r requirements.txt
```
Launch ComfyUI.

Note
Users must return to the home directory of the repository in order to run main.py.
```
python main.py
```
Click the URL to open ComfyUI in a browser.

Note
After running the above command, ensure that the Pytorch version seen in the terminal is the ROCm 7.2 specific one.

Note
If running on low-memory configs, try adding the --lowvram and --disable-pinned-memory parameters to the run command.
From the ComfyUI frontend, click on Templates → Search for ‘SDXL Turbo’ → Follow prompt for model install and place it into the ‘checkpoints’ folder under ComfyUI>models>checkpoints.
From the ComfyUI frontend, click ‘Run’ to begin image generation.

Compatibility matrices#

This section provides information on the compatibility of ROCm™ components, Radeon™ GPUs, and the Radeon Software for Linux® version (Kernel Fusion Driver) and Windows Subsystem for Linux (WSL).

Note

To rollback support matrices and install instructions for previous versions, click Version List located at the top-right corner of the screen, or select the version (v:) menu on the bottom-left.

Linux Compatibility

Linux support matrices by ROCm version#

ROCm 7.2#

Compatible OS, GPU, and framework support matrices for the latest ROCm release.

To rollback support matrices and install instructions for previous versions, click Version List located at the top-right corner of the screen, or select the version (v:) menu on the bottom-left.

OS support matrix#

OS	Kernel	Supported
Ubuntu® 24.04.3 Desktop Version with HWE	Ubuntu kernel 6.14	Yes
Ubuntu® 22.04.5 Desktop Version with HWE	Ubuntu kernel 6.8	Yes
RHEL 10.1	Linux kernel 6.12	Yes

Note
AMD currently provides preliminary support for Ubuntu 24.04.3 using the Ubuntu 24.04.2 installer.

GPU support matrix#

ROCm Version	Radeon™ Software for Linux® Version	Supported AMD Radeon™ Hardware
7.2	22.04 24.04 25.10	AMD Radeon RX 9070 AMD Radeon RX 9070 XT AMD Radeon RX 9070 GRE AMD Radeon AI PRO R9700 AMD Radeon RX 9060 AMD Radeon RX 9060 XT AMD Radeon RX 7900 XTX AMD Radeon RX 7900 XT AMD Radeon RX 7900 GRE AMD Radeon PRO W7900 AMD Radeon PRO W7900 Dual Slot AMD Radeon PRO W7800 AMD Radeon PRO W7800 48GB AMD Radeon RX 7800 XT AMD Radeon PRO W7700 AMD Radeon RX 7700 AMD Radeon RX 7700 XT AMD Radeon AI PRO R9600D AMD Radeon RX 9060

Framework + ROCm support matrices#

View the ROCm support matrices for PyTorch, ONNX, and TensorFlow frameworks.

PyTorch + ROCm support matrix#

PyTorch Version	ROCm Version	Comments
2.9.1	7.2	Official production support. See Install PyTorch for Radeon GPUs.
2.9/Nightly	7.2	Available from PyTorch.org nightly builds, not tested extensively by AMD.
2.7/Stable	7.2	Not supported for Radeon 7000 series.

AI Data Types

FP32

FP16

Mixed precision (FP32/FP16)

INT8

ONNX Runtime + ROCm support matrix#

ONNX RT Version	ROCm Version	Comments
1.23.1	7.2	Official production support. See Install ONNX for Radeon GPUs.

AI Data Types

FP32

FP16

INT8 (MIGraphX)

Mixed precision (FP32/FP16)

Note Refer to Installation Instructions to Get Started with ONNX Runtime for more information.

TensorFlow + ROCm support matrix#

TensorFlow Version	ROCm Version	Comments
2.20	7.2	Official production support. See Install TensorFlow for Radeon GPUs.

AI Data Types

FP32

FP16

Triton + ROCm support matrix#

Triton Version	ROCm Version	Comments
3.5.1	7.2	Official production support. See Install Triton for Radeon GPUs.

Note Refer to the official Triton documentation for more information.

Docker support matrix#

See Docker Image Support Matrix for the latest version of the software support matrices for ROCm container releases.

WSL support matrices by ROCm version#

ROCm 7.2#

This section provides information on the compatibility of ROCm™ components, Radeon™ GPUs, and the Radeon Software for Windows Subsystem for Linux® (WSL).

To rollback support matrices and install instructions for previous versions, click Version List located at the top-right corner of the screen, or select the version (v:) menu on the bottom-left.

OS support matrix#

OS	Kernel	Supported
Ubuntu® 24.04.2 Desktop Version with HWE	WSL2-Linux-Kernel 5.15	Yes
Ubuntu 22.04 or Ubuntu 22.04 LTS	WSL2-Linux-Kernel 5.15	Yes

Note
Refer to How to install Linux on Windows with WSL for up-to-date OS compatibility information.

GPU support matrix#

ROCm Version	Radeon™ Software for Linux® Version	Radeon™ Software for Windows Version	Supported AMD Radeon™ Hardware
7.2	22.04 24.04 25.10.1 for WSL	AMD Software: Adrenalin Edition™ 26.1.1 for WSL2	AMD Radeon RX 9070 AMD Radeon RX 9070 XT AMD Radeon RX 9070 GRE AMD Radeon AI PRO R9700 AMD Radeon RX 9060 AMD Radeon RX 9060 XT AMD Radeon RX 7900 XTX AMD Radeon RX 7900 XT AMD Radeon RX 7900 GRE AMD Radeon PRO W7900 AMD Radeon PRO W7900 Dual Slot AMD Radeon PRO W7800 AMD Radeon PRO W7800 48GB AMD Radeon RX 7800 XT AMD Radeon PRO W7700 AMD Radeon RX 7700 AMD Radeon RX 7700 XT AMD Radeon AI PRO R9600D AMD Radeon RX 9060

Framework + ROCm support matrices#

View the ROCm support matrices for PyTorch, ONNX, and TensorFlow frameworks.

PyTorch + ROCm support matrix#

PyTorch Version	ROCm Version	Comments
2.9.1	7.2	Official production support. See Install PyTorch for Radeon GPUs on WSL.
2.9/Nightly	7.2	Available from PyTorch.org nightly builds, not tested extensively by AMD.
2.7/Stable	7.2	Not supported for Radeon 7000 series.

AI Data Types

FP32

FP16

Mixed precision (FP32/FP16)

INT8

ONNX + ROCm support matrix#

ONNX Version	ROCm Version	Comments
1.23.1	7.2	Official production support. See Install ONNX for Radeon GPUs on WSL.

AI Data Types

FP32

FP16

INT8 (MIGraphX)

Mixed precision (FP32/FP16)

Note Refer to Installation Instructions to Get Started with ONNX Runtime for more information.

TensorFlow + ROCm support matrix#

TensorFlow Version	ROCm Version	Comments
2.20	7.2	Official production support. See Install TensorFlow for Radeon GPUs on WSL.

AI Data Types

FP32

FP16

Triton + ROCm support matrix#

Triton Version	ROCm Version	Comments
3.5.1	7.2	Official production support. See Install Triton for Radeon GPUs on WSL.

Note Refer to the official Triton documentation for more information.

Docker support matrix#

See Docker Image Support Matrix for the latest version of the software support matrices for ROCm container releases.

Windows support matrices by ROCm version#

PyTorch on Windows#

Compatible OS, GPU, and framework support matrices for the latest ROCm release.

To rollback support matrices and install instructions for previous versions, click Version List located at the top-right corner of the screen, or select the version (v:) menu on the bottom-left.

OS support matrix#

OS	Supported
Windows 11	Yes

GPU support matrix#

ROCm Version	Supported Architectures	Supported AMD Radeon™ Hardware
7.2*	gfx1201 gfx1200 gfx1100 gfx1101	AMD Radeon RX 9070 AMD Radeon RX 9070 XT AMD Radeon AI PRO R9700 AMD Radeon RX 9060 XT AMD Radeon RX 7900 XTX AMD Radeon PRO W7900 AMD Radeon PRO W7900 Dual Slot AMD Radeon RX 7700

Framework + ROCm support matrices#

View the ROCm support matrices for PyTorch.

PyTorch + ROCm support matrix#

PyTorch Version	ROCm Version	Python Version	Comments
2.9	7.2*	3.12	See Install PyTorch for Radeon GPUs.

AI Data Types

FP16

FP8 (Supported only on RDNA4 GPUs)

*Pytorch on Windows includes ROCm 7.2 components; however, the entire ROCm stack is not yet supported on Windows.

Compatibility matrices#

This section provides information on the compatibility of ROCm™ components, Ryzen™ APUs for Linux and Windows.

Note

To rollback support matrices and install instructions for previous versions, click Version List located at the top-right corner of the screen, or select the version (v:) menu on the bottom-left.

Linux Compatibility

Linux support matrices by ROCm version#

ROCm 7.2#

Compatible OS, GPU, and framework support matrices for the latest ROCm release.

To rollback support matrices and install instructions for previous versions, click Version List located at the top-right corner of the screen, or select the version (v:) menu on the bottom-left.

OS support matrix#

OS	Supported
Ubuntu® 24.04.3	Yes

Note
AMD currently provides preliminary support for Ubuntu 24.04.3 using the Ubuntu 24.04.2 installer.

GPU support matrix#

ROCm Version	Supported Architectures	Supported AMD Radeon™ Hardware
7.2	gfx1150 gfx1151	AMD Ryzen AI Max+ 395 AMD Ryzen AI Max 390 AMD Ryzen AI Max 385 AMD Ryzen AI 9 HX 375 AMD Ryzen AI 9 HX 370 AMD Ryzen AI 9 365 AMD Ryzen AI 9 HX 475 AMD Ryzen AI 9 HX 470 AMD Ryzen AI 9 465

Framework + ROCm support matrices#

View the ROCm support matrices for PyTorch.

PyTorch + ROCm support matrix#

PyTorch Version	ROCm Version	Python Version	Comments
2.9	7.2	3.12	Official production support. See Install PyTorch for Ryzen APUs.

AI Data Types

FP16

Note: Only FP16 has been officially validated, but other data types may work as well.

Windows support matrices by ROCm version#

PyTorch on Windows#

Compatible OS, GPU, and framework support matrices for the latest ROCm release.

To rollback support matrices and install instructions for previous versions, click Version List located at the top-right corner of the screen, or select the version (v:) menu on the bottom-left.

OS support matrix#

OS	Supported
Windows 11	Yes

GPU support matrix#

ROCm Version	Supported Architectures	Supported AMD Radeon™ Hardware
7.2*	gfx1150 gfx1151	AMD Ryzen AI Max+ 395 AMD Ryzen AI Max 390 AMD Ryzen AI Max 385 AMD Ryzen AI 9 HX 375 AMD Ryzen AI 9 HX 370 AMD Ryzen AI 9 365 AMD Ryzen AI 9 HX 475 AMD Ryzen AI 9 HX 470 AMD Ryzen AI 9 465

Framework + ROCm support matrices#

View the ROCm support matrices for PyTorch.

PyTorch + ROCm support matrix#

PyTorch Version	ROCm Version	Python Version	Comments
2.9	7.2*	3.12	See Install PyTorch for Ryzen APUs.

AI Data Types

FP16

*Pytorch on Windows includes ROCm 7.2 components; however, the entire ROCm stack is not yet supported on Windows.

Radeon Limitations and recommended settings#

This section provides information on software and configuration limitations.

Note
For ROCm on Instinct known issues, refer to AMD ROCm Documentation

For OpenMPI limitations, see ROCm UCX OpenMPI on Github

7.2 release known issues#

Linux#

Known issues#

Visual corruption and abnormal colors may be observed while generating videos using the ComfyUI Wan 2.2 TI2V 5B model on some AMD Graphics Products, such as the Radeon™ RX 9070 GRE and Radeon™ AI PRO R9700.
Intermittent errors or segmentation faults may occur while running JAX workloads.
Failures or instability may be observed while running SD3.5XL or FLUX inference workloads on configs with lower system memory (e.g., 32 GB). Users experiencing this issue are recommended to try the --lowvram and --disable-pinned-memory parameters in the run command.

Limitations#

AO Triton with PyTorch 2.9 is disabled by default for AMD Radeon RX 7000 series graphics products, and must be enabled manually.

Multi-GPU configuration#

AMD has identified common errors when running ROCm™ on Radeon™ multi-GPU configuration at this time, along with the applicable recommendations.

See mGPU known issues and limitations for a complete list of mGPU known issues and limitations.

Windows#

Note
The following Windows known issues and limitations are applicable to the 7.2 release. Only Pytorch is currently available on Windows - the rest of the ROCm stack is only supported on Linux.
AMD is aware and actively working on resolving these issues for future releases.

Note
If you encounter errors related to missing .dll libraries, install Visual C++ 2015-2022 Redistributables.

Known issues#

Disable the following Windows security features as they can interfere with ROCm functionality:
- Turn off WDAG (Windows Defender Application Guard)
  - Control Panel > Programs > Programs and Features > Turn Windows features on or off > Clear “Microsoft Defender Application Guard”
- Turn off SAC (Smart App Control)
  - Settings > Privacy & security > Windows Security > App & browser control > Smart App Control settings > Off

Limitations#

No ML training support.
Only Python 3.12 is supported.
For ComfyUI, adding the --lowvram and --disable-pinned-memory parameters may help with lower-memory configs.
On Windows, only Pytorch is supported, not the entire ROCm stack.
On Windows, the latest version of transformers should be installed, via pip install. Some older versions of transformers (<4.55.5) might not be supported.
On Windows, only LLM batch sizes of 1 are officially supported.

WSL#

Known issues#

Intermittent script failure may be observed while running Llama 3 inference workloads with vLLM in WSL2. End users experiencing this issue are recommended to follow vLLM setup instructions here.
Intermittent script failure or driver timeout may be observed while running Stable Diffusion 3 inference workloads with JAX.
Lower than expected performance may be observed while running inference workloads with JAX in WSL2.
Intermittent script failure may be observed while running Resnet50, BERT, or InceptionV3 training workloads with ONNX runtime.
Output error message (resource leak) may be observed while running Llama 3.2 workloads with vLLM.
Output error message (VaMgr) may be observed while running PyTorch workloads in WSL2.
Intermittent script failure or driver timeout may be observed while running Stable Diffusion inference workloads with TensorFlow.
Intermittent application crash may be observed while running Stable Diffusion workloads with ComfyUI and MIGraphX on Radeon™ RX 9060 series graphics products.
Intermittent script failure may occur while running Stable Diffusion 2 workloads with PyTorch and MIGraphX
Intermittent script failure may occur while running LLM workloads with PyTorch on Radeon™ PRO W7700 graphics products.
Lower than expected performance (compared to native Linux) may be observed while running inference workloads (eg. Llama2, BERT) in WSL2.

Important!
Radeon™ PRO Series graphics cards are not designed nor recommended for datacenter usage. Use in a datacenter setting may adversely affect manageability, efficiency, reliability, and/or performance. GD-239.

Important!
ROCm is not officially supported on any mobile SKUs.

WSL recommended settings#

Optimizing GPU utilization
WSL overhead is a noted bottleneck for GPU utilization. Increasing the batch size of operations will load the GPU more optimally, reducing time required for AI workloads. Optimal batch sizes vary by model, and macro-parameters.

ROCm support in WSL environments#

AMD-smi support

Due to WSL architectural limitations for native Linux User Kernel Interface (UKI), amd-smi is not supported.

Issue	Limitations
UKI does not currently support amd-smi	No current support for: Active compute processes GPU utilization Modifiable state features

ROCm-profiler support

Not currently supported.

Debugger

Not currently supported.

Running PyTorch in virtual environments

Running PyTorch in virtual environments requires a manual libhsa-runtime64.so update.

When using the WSL usecase and hsa-runtime-rocr4wsl-amdgpu package (installed with PyTorch wheels), users are required to update to a WSL compatible runtime lib.

Solution:

Enter the following commands:

location=`pip show torch | grep Location | awk -F ": " '{print $2}'`
cd ${location}/torch/lib/
rm libhsa-runtime64.so*
cp /opt/rocm/lib/libhsa-runtime64.so.1.2 libhsa-runtime64.so

Ryzen Limitations and recommended settings#

This section provides information on software and configuration limitations.

7.2 release#

Linux#

Known issues#

Failures or instability may be observed while running SD3.5XL or FLUX inference workloads on configs with lower system memory (e.g., 32 GB). Users experiencing this issue are recommended to try the --lowvram and --disable-pinned-memory parameters in the run command.

Windows#

Note
The following Windows known issues and limitations are applicable to the 7.2 release. Only Pytorch is currently available on Windows - the rest of the ROCm stack is only supported on Linux.
AMD is aware and actively working on resolving these issues for future releases.

Note
If you encounter errors related to missing .dll libraries, install Visual C++ 2015-2022 Redistributables.

Known issues#

Disable the following Windows security features as they can interfere with ROCm functionality:
- Turn off WDAG (Windows Defender Application Guard)
  - Control Panel > Programs > Programs and Features > Turn Windows features on or off > Clear “Microsoft Defender Application Guard”
- Turn off SAC (Smart App Control)
  - Settings > Privacy & security > Windows Security > App & browser control > Smart App Control settings > Off

Limitations#

No ML training support.
Only Python 3.12 is supported.
Only Pytorch is supported, not the entire ROCm stack.
The latest version of transformers should be installed, via pip install. Some older versions of transformers (<4.55.5) might not be supported.
The torch.distributed module is currently not supported. Some functions from diffusers and accelerate module may get affected.
For ComfyUI, adding the --lowvram and --disable-pinned-memory parameters may help with lower-memory configs.
On Linux, if you get an error regarding convolutions while running Generative AI workloads, setting this environment variable to 1 may help: MIOPEN_DEBUG_CONV_DIRECT_NAIVE_CONV_FWD
On Windows, only LLM batch sizes of 1 are officially supported.

AI community#

Want to share your experiences, find answers, or contribute to resolving issues?

Explore the AMD AI Community Forum, where you will find a like-minded community, passionate about all things AI!

Report a bug#

Found a defect? Report issues through ROCm GitHub Issues, and contribute to improving our user experience.

Note
Windows users can also submit issues to ROCm GitHub Issues.

Selection	Prefill	Decode	Flags
TRITON_ATTN (default)	`kernel_unified_attention`	`same unified`	None or `--attention-config.backend TRITON_ATTN`
ROCM_ATTN (custom paged attention)	`context_attention_fwd`	`paged_attention_rocm`	`VLLM_V1_USE_PREFILL_DECODE_ATTENTION=1` or `--attention-config.backend ROCM_ATTN --attention-config.use_prefill_decode_attention=true`
ROCM_AITER_UNIFIED_ATTN	`AITER unified_attention`	`same unified`	`VLLM_ATTENTION_BACKEND=ROCM_AITER_UNIFIED_ATTN` or `--attention-config.backend ROCM_AITER_UNIFIED_ATTN`

Use ROCm on Radeon and Ryzen

Contents

Use ROCm on Radeon and Ryzen#

Expanded Platform Support#

Your Local, Private AI Powerhouse#

Leadership in Open-Source GPU Programming#

ROCm™ Key Capabilities#

Get Started#

Prerequisites to use ROCm on Radeon desktop GPUs for machine learning development#

Supported hardware#

Supported graphics processing units#

Recommended memory#

Minimum recommendations#

Supported operating systems#

Update Ubuntu® operating system#

Recommended system configuration#

PCIe atomics for PyTorch#

Disable iGPU#

Gigabyte™ X670 AORUS ELITE AX#

ASUS Prime X670-P WIFI#

Alternative option: Use environment variables to select target GPU#

Additional package repositories#

Red Hat Enterprise Linux#

Prerequisites to use ROCm on Ryzen APUs for machine learning development#

Supported Hardware#

Supported Graphics Processing Units#

Recommended memory#

Supported Operating Systems#

Update Ubuntu® Operating System#

Linux How to guide - Use ROCm on Radeon and Ryzen#

Install Radeon software for Linux with ROCm#

Install AMD unified driver package repositories and installer script#

Install AMD unified kernel-mode GPU driver, ROCm, and graphics#

Graphics usecase#

Set Groups permissions#

Add user to render and video groups#

Post-install verification checks#

Advanced install methods#

Uninstall ROCm#

Upgrade to newer versions of Radeon software for Linux#

Install PyTorch for ROCm#

Install methods#

Option A: PyTorch via PIP installation#

Option B: Docker installation#

Prerequisites to install PyTorch using Docker#

Use Docker image with pre-installed PyTorch#

Verify PyTorch installation#

Install ONNX Runtime for Radeon GPUs#

Overview#

Prerequisites#

Verify MIGraphX installation#

Install ONNX Runtime#

Verify ONNX Runtime installation#

Install TensorFlow for ROCm#

Pre-requisites#

PIP installation#

Verify TensorFlow installation#

Run basic TensorFlow example#

Install Triton for ROCm#

Pre-requisites#

Install libraries#

Install JAX for ROCm#

Install JAX#

PIP installation#

Docker installation#

Verify installation#

Install MIGraphX for Radeon GPUs#

Introduction to MIGraphX#

Prerequisites#

Install MIGraphX#

Install Torch-MIGraphX#

Verify Torch-MIGraphX installation#

Install and verify MIGraphX for ONNX Runtime#

mGPU setup and configuration#

Hardware and software considerations#

Hardware considerations#

Software considerations#

mGPU configuration by framework#

PyTorch#

mGPU known issues and limitations#

`amd-ttm` usage examples#