Two primary benchmark tracks: core 5-property strict holdouts (S2/S3) with external apples-to-apples comparisons, and universal 16-property validation with strict and out-of-family stress tests. This page also includes a curated mini-benchmark for defect-context gemstone color behavior.
Deterministic production runtime. No ML inference in final calculations.
| Test | Overall MAPE | Bulk Modulus (B) | Debye Temp (theta_D) | Sound Velocity | Density | Thermal Cond. (kappa) | Runtime mean |
|---|---|---|---|---|---|---|---|
| Test A: Family holdout (S2) | 1.1668% | 0.9056% | 1.3283% | 1.4405% | 1.2231% | 0.9185% | 0.3294 ms |
| Test B: Interaction holdout (S3) | 1.3750% | 1.0149% | 1.5636% | 1.6081% | 1.3398% | 1.3473% | 0.3205 ms |
Units: bulk modulus (GPa), Debye temperature (K), sound velocity (m/s), density (g/cm^3), thermal conductivity (W/(m*K)).
Strict reference validation plus out-of-family stress tests across 15 split scenarios.
| Property | Strict MAPE % (N) | Out-of-family weighted MAPE % (rows) |
|---|---|---|
| Crystal structure | 0.000 (59) | 0.000 (59) |
| Lattice constant (A) | 0.085 (59) | 0.086 (59) |
| Lattice constant (pm) | 0.085 (59) | 0.086 (59) |
| Atomic volume (A^3) | 0.497 (71) | 0.488 (71) |
| Band gap (eV) | 0.690 (274) | 0.985 (674) |
| Optical band gap (eV) | 0.527 (39) | 0.530 (39) |
| Dielectric constant | 0.910 (32) | 0.925 (32) |
| Refractive index | 0.658 (49) | 0.662 (49) |
| Reflectivity | 0.339 (28) | 0.338 (28) |
| Hardness (GPa) | 0.740 (44) | 0.724 (44) |
| Cv (J/mol*K) | 0.857 (39) | 0.858 (39) |
| Cv (J/kg*K) | 0.857 (39) | 0.858 (39) |
| Cp (J/mol*K) | 0.797 (81) | 0.801 (81) |
| Thermal expansion (1/K) | 0.903 (39) | 0.908 (39) |
| Melting point (K) | 0.947 (82) | 0.973 (82) |
| Density (g/cm^3) | 0.929 (162) | 0.965 (165) |
For crystal structure, values are mismatch-rate percentages (0% means all evaluated rows matched).
All models below are scored on the same strict splits. Lower MAPE is better.
| Model | Overall | Bulk Modulus | Debye Temp | Sound Velocity | Density | Thermal Cond. |
|---|---|---|---|---|---|---|
| FLUX | 1.1668% | 0.9056% | 1.3283% | 1.4405% | 1.2231% | 0.9185% |
| AFLOW adapter | 36.0693% | 14.3580% | 15.6790% | 29.5091% | 5.5695% | 122.9277% |
| JARVIS adapter | 10.9230% | 15.4191% | NA | NA | 6.3425% | NA |
| Matbench adapter | 18.4214% | 18.4214% | NA | NA | NA | NA |
| Model | Overall | Bulk Modulus | Debye Temp | Sound Velocity | Density | Thermal Cond. |
|---|---|---|---|---|---|---|
| FLUX | 1.3750% | 1.0149% | 1.5636% | 1.6081% | 1.3398% | 1.3473% |
| AFLOW adapter | 35.3566% | 14.5881% | 14.1047% | 30.0588% | 5.5493% | 119.6271% |
| JARVIS adapter | 10.9356% | 15.4816% | NA | NA | 6.2780% | NA |
| Matbench adapter | 18.4247% | 18.4247% | NA | NA | NA | NA |
Important: NA means that model did not provide that property in this comparison. Coverage differs by model and property.
A curated showcase of defect-center and illumination behavior using the same deterministic universal runtime path.
| Case | Context | Expected | Predicted |
|---|---|---|---|
| Amethyst | Quartz + Fe center | Purple | Purple |
| Ruby | Corundum + Cr center | Red | Red |
| Emerald | Beryl + Cr/V center | Green | Green |
| Blue Topaz | Topaz + F-like center | Blue | Blue |
| Alexandrite (daylight) | Cr center, daylight profile | Green | Green |
| Alexandrite (incandescent) | Same material, warm-light profile | Red | Red |
Runtime remains deterministic and constant-time. No ML inference is used in final engine calculations.
Core track: 5 thermo-mechanical properties. Universal track: 16 properties spanning structural, electronic, optical, thermal, and mechanical outputs.
MAPE (%). Lower is better.
Mean runtime in milliseconds per prediction call.
The gemstone color section is a curated feature benchmark that demonstrates defect and illumination controls. It is separate from core strict property claims.
Both tests are designed so the engine must generalize beyond familiar patterns.
All materials from one family are held out together. This checks whether the model extrapolates to unseen families.
Bonding interaction types are held out by group. This checks robustness across unseen interaction patterns.
Both benchmark tracks are frozen to public snapshots so every number can be rerun and verified.
Dirty: true indicates public report-formatting/export changes after the prediction rows were frozen. The benchmark prediction rows, snapshot ID, commit, commands, and attached hashes define the reproducible artifact.
Specific property accuracy and methodology details are reported on dedicated pages with their own datasets and metric definitions.
This page aggregates many materials properties. The aggregate basis is mixed; property rows and source notes identify the relevant prediction route for each result family.