Troubleshoot devices & SDK
Hardware problems usually come down to the cloud boundary, a missing card, a release mismatch, a stale model artifact, a broken install, a stale driver, or a crashing pipeline. Find your symptom, then apply the matching fix.
Collect the evidence
Before changing anything, collect the evidence. These read-only commands answer most questions on their own, and every fix below starts from them.
bash
axversion # SDK version
axversion --driver # driver version
axdevice -v # firmware + board controller
lspci | grep -i axelera
ls -l /dev/metis-*
lsmod | grep metisSymptoms and fixes
| Symptom | Fix |
|---|---|
Chat says my device list is empty, but axdevice sees my Axelera card. | That is expected: hosted Chat runs in the cloud and cannot see your computer, your Voyager SDK install, or /dev/metis devices. Your local setup is fine. Open the desktop app and use Agent Dev for anything that touches the card; keep Chat for planning and knowledge. |
| Agent Dev posted “Voyager SDK was not detected on this machine.” | The one-time notice means no SDK was found in the standard locations. Agent Dev still works for general coding; install the SDK from the linked guide (see Set up the Voyager SDK & your card), or export AXELERA_FRAMEWORK pointing at an existing install. Wingman re-checks automatically. |
axdevice lists nothing. | Check lspci | grep -i axelera first. If the card is absent there, the host itself does not see it — check seating, power, and BIOS PCIe settings before any software step. If it is present, check the selected release’s axdevice reference before trying axdevice --refresh, then inspect /dev/metis-* permissions and confirm that lsmod | grep metis shows the driver. axdevice --reboot power-cycles the card as a last resort — it kills any in-flight inference. |
Card is listed, but axdevice or inference fails with ENOTTY or AXL_IOCTL errors. | Driver and runtime are out of step — this is the classic post-upgrade trap. Upgrade metis-dkms to the driver matching your runtime, then redeploy previously compiled models: payloads built against the old runtime must be rebuilt with deploy.py or refetched with axdownloadmodel. |
| Versions look mismatched. | Read the compatibility matrix from the SDK release you selected instead of reusing an old version pin. In the selected v1.8.0 snapshot, Linux driver 1.6.2 is required, firmware 1.8.0 is recommended (1.7.0, 1.6.0, 1.5.0, and 1.4.0 are supported), and board controller 7.4 is recommended (7.0 supported). axdevice driver --install selects that release’s recommended driver. axdevice interactive_flash_update changes hardware firmware, so never run it alongside other device work and do not interrupt it. |
| Inference says the model version is unsupported after an SDK upgrade. | The cached artifact may belong to an older AxModel major. The selected Voyager SDK v1.8.0 release uses AxModel major 5 and rejects artifacts from earlier AxModel majors. Refresh a prebuilt model with axdownloadmodel <model> --force, or rebuild a custom model through the YAML, recipe, or AxScript surface that owns it. Check the selected release’s compatibility matrix before applying this diagnosis to another version. |
| The safe model inspector returned only a hash or says a parser is unavailable. | That is a successful bounded inspection, not a compile result. The inspector deliberately avoids model-code imports, network access, inference, compilation, ONNX shape inference, external ONNX tensor data, and unsafe PyTorch pickle loading. Use the file type and hash to continue discovery, then collect architecture, input, output, preprocessing, and licensing facts from trusted model documentation. |
| I passed a bare ONNX file to AxScript. | Choose a compatible YAML model or standalone recipe instead. AxScript does not import ONNX: it needs compatible weights or a checkpoint plus a source implementation, reference architecture, authored device graph, and any required kernels. A new architecture belongs in model-porting work before python -m model_axscript build <tag> can apply. |
| An AxScript card built successfully, but its status is pending. | Keep it pending. A build proves artifact creation only. Run it on Axelera hardware and apply the model card’s reference and accuracy gates before calling it validated. Pending cards are useful for bring-up, but they are not downloadable prebuilt payloads or completed deployments. |
A published precompiled LLM fails under axllm. | Confirm the exact selected-release model-zoo entry, payload download, context and memory envelope, AxModel/runtime compatibility, and the documented Experimental prompt command. This is a runtime path for published weights; it does not require or prove custom compilation support. |
| I need to port a custom LLM, LSTM, GRU, RNN, or another architecture. | Require a complete registered recipe or family driver, the current operator support table, active compiler or wheel, produced artifact, bounded hardware run, and model-specific quality comparison. Do not generalize from a published precompiled LLM, reuse vision AxScript commands, or publish preview catalog entries as released custom-build availability. Until that exact-release evidence agrees, keep the route unverified. |
| SDK install is missing, old, or broken. | Follow the fresh-install path on Set up the Voyager SDK & your card. Keep one virtual environment per SDK version — do not upgrade packages inside an old venv in place. |
| A pipeline crashes or misbehaves mid-run. | Check the run log first — packaged launches keep it at results/inference-*.log. For GStreamer-level detail, rerun with GST_DEBUG=3. The fastest route is handing the log to Agent Dev: “this pipeline crashed, here is the log, find the root cause.” |
| The Agent Dev readiness control expands to Fix now. | Click Fix now. It walks through the exact fix: installing the missing CLI, signing in to it, or approving the Wingman connection in your browser. Only a compact green dot proves the selected provider is ready. Compact amber checking or degraded dots are diagnostic states, not proof of readiness; click one for details or a retry. |