about [ai] physics
Mission, consortium governance, and how we evaluate models on the platform.
Mission
AI Physics benchmarks are fragmented. Research groups use different datasets, metrics, and evaluation protocols, making it hard to compare models fairly or know which surrogates are trustworthy for engineering decisions.
AIPhysics.org provides transparent, standardized evaluation focused on what engineers actually care about: predicted vs. actual values on decision-driving quantities like lift and drag coefficients, peak stress, and stress concentration factors.
How we score
Rankings on the leaderboard combine two signals per model: global scalar accuracy (Cl, Cd, Kt, peak stress, etc.) and nodal field R² on simulation mesh outputs. The combined score is the geometric mean of these averages. A model must perform well on both scalars and fields to rank highly.
Domain scores average across benchmarks within Structural, Fluid dynamics, and other domains. Training and inference times are shown for context but are not used for ranking, since hardware varies widely.
Consortium
Early-stage community working toward shared benchmarks, metrics, and evaluation standards.
The consortium is in early stages. AIPhysics.org is operated by Rescale AI Engineering today. The outline below is where we are headed as partners get involved, not how the platform is run right now.
- Consortium council (planned): Equal representation from industry and academic members. Quarterly review of benchmarks and metrics.
- Metrics working groups (planned): Domain experts would define evaluation metrics and engineering thresholds for each benchmark.
- Transparent process (planned): Decisions documented publicly. Community input welcomed on benchmark proposals.
Contact
Consortium membership, benchmark questions, and technical support.
Email us at consortium@aiphysics.org