Cross-Compiler Benchmarking on IBM Heron: Q-Psi vs Qiskit O3 and TKET
We evaluate the Q-Psi State-Space Compiler against industrial compiler toolchains Qiskit O3 and TKET across the standard MQT Bench suite on IBM Heron (ibm_marrakesh, 156 programmable qubits, Job da1ul7e3kjvs73877tag, 36,864 shots). Q-Psi compiled all 18 benchmark circuits into valid QPU executables. Physical fidelity measurements demonstrated workload-dependent competitiveness: Qiskit O3 held a small aggregate advantage (mean delta \(-0.0067\)), while Q-Psi's apparent aggregate advantage over TKET (\(+0.0398\)) was driven entirely by a severe routing failure in TKET on vqe_su2_n8 (\(+0.2696\) delta). Omitting vqe_su2_n8 shifts the Q-Psi vs. TKET delta to \(-0.0062\).
1. Experimental Setup & Workload Suite
The benchmark suite comprised 6 standard algorithmic families from the Munich Quantum Toolkit (MQT Bench): dj_n6, ghz_n8, qft_n6, graphstate_n8, vqe_su2_n8, and wstate_n6. Each circuit was compiled by Q-Psi, Qiskit (Optimization Level 3), and TKET (Default Routing), producing 18 physical QPU circuits evaluated with 2,048 shots each.
2. Physical Fidelity Comparison
| Circuit | Q-Psi Fidelity | Qiskit O3 Fidelity | TKET Fidelity | Δ(Q-Psi - Qiskit) | Δ(Q-Psi - TKET) |
|---|---|---|---|---|---|
| dj_n6 | 0.9612 | 0.9645 | 0.9630 | -0.0033 | -0.0018 |
| ghz_n8 | 0.8840 | 0.8912 | 0.8890 | -0.0072 | -0.0050 |
| qft_n6 | 0.7120 | 0.7315 | 0.7240 | -0.0195 | -0.0120 |
| graphstate_n8 | 0.8410 | 0.8450 | 0.8435 | -0.0040 | -0.0025 |
| vqe_su2_n8 | 0.6845 | 0.6810 | 0.4149 | +0.0035 | +0.2696 |
| wstate_n6 | 0.7950 | 0.8045 | 0.8050 | -0.0095 | -0.0100 |
3. Leave-One-Out Sensitivity Analysis
The sensitivity audit confirms that Q-Psi does not hold an aggregate advantage over TKET across general circuits; the positive delta was an artifact of TKET's heavy-hex routing layout on vqe_su2_n8.
4. Conclusion & Boundary
Q-Psi demonstrates valid compilation and competitive execution across industrial benchmarks, but does NOT establish general superiority over Qiskit or TKET.