Rust ONNX Runtime for AMD GPUs
Windows driver and runtime limitations kept AMD GPUs from running a broad range of models locally. RemSoft.Dev developed custom Rust software to run ONNX models on AMD GPUs through WSL/ROCm, overcoming those limitations. For a local RAG workload, the engine parallelizes execution across a powerful discrete GPU and an integrated GPU: indexing stays on the primary GPU while a second RAG instance serves search on the integrated GPU. Search remains available during indexing, hardware utilization improves, and the platform can run a broad range of Hugging Face ONNX models instead of being limited to GGUF/Llama formats. The same Rust layer powers local model execution in Dew Flow, our multi-agent development orchestrator.