about [ai] physics
Mission, consortium governance, and how we evaluate models on the platform.
Mission
AI Physics benchmarks are fragmented. Research groups use different datasets, metrics, and evaluation protocols, making it hard to compare models fairly or know which surrogates are trustworthy for engineering decisions.
AIPhysics.org provides transparent, standardized evaluation focused on what engineers actually care about: predicted vs. actual values on decision-driving quantities like lift and drag coefficients, peak stress, and stress concentration factors.
Where we lean on open community datasets, notably the OpenRadioss Neon crash HPC model and DrivAerML from the CAE ML Datasets project (with simulation and dataset work associated with NVIDIA), we credit those projects and point people to their upstream downloads for the full models and corpora. See the OpenRadioss and DrivAerML attribution FAQs.
How we score
Rankings on the leaderboard combine two signals per model when both are available: global scalar accuracy (Cl, Cd, Kt, peak stress, etc.) and nodal field R² on simulation mesh outputs. When both exist, the combined score is the geometric mean of these averages, so a model must perform well on both scalars and fields to rank highly. When only nodal or only global scores are published, the headline score is that score alone.
Domain scores average within Structural, Fluid 2D (airfoil), Fluid 3D (DrivAerML), and Crash. Fluid 2D and Fluid 3D stay separate so a strong small-mesh aero score cannot stand in for native large-mesh vehicle aero. Training and inference times are shown for context but are not used for ranking, since hardware varies widely.
Consortium
Early-stage community working toward shared benchmarks, metrics, and evaluation standards.
The consortium is in early stages. AIPhysics.org is operated by Rescale AI Engineering today. The outline below is where we are headed as partners get involved, not how the platform is run right now.
- Consortium council (planned): Equal representation from industry and academic members. Quarterly review of benchmarks and metrics.
- Metrics working groups (planned): Domain experts would define evaluation metrics and engineering thresholds for each benchmark.
- Transparent process (planned): Decisions documented publicly. Community input welcomed on benchmark proposals.
Contact
Consortium membership, benchmark questions, and technical support.