Two primary benchmark tracks: core 5-property strict holdouts (S2/S3) with external apples-to-apples comparisons, and universal Layer 7 validation across 16 properties with strict and out-of-family stress tests. This page also includes a curated mini-benchmark for defect-context gemstone color behavior.
Core track: 5 thermo-mechanical properties. Universal track: 16 properties spanning structural, electronic, optical, thermal, and mechanical outputs.
MAPE (%). Lower is better.
Mean runtime in milliseconds per prediction call.
The gemstone color section is a curated feature benchmark that demonstrates defect and illumination controls. It is separate from core strict property claims.
Both tests are designed so the engine must generalize beyond familiar patterns.
All materials from one family are held out together. This checks whether the model extrapolates to unseen families.
Bonding interaction types are held out by group. This checks robustness across unseen interaction patterns.
Deterministic production runtime. No ML inference in final calculations.
| Test | Overall MAPE | Bulk Modulus (B) | Debye Temp (theta_D) | Sound Velocity | Density | Thermal Cond. (kappa) | Runtime mean |
|---|---|---|---|---|---|---|---|
| Test A: Family holdout (S2) | 1.1668% | 0.9056% | 1.3283% | 1.4405% | 1.2231% | 0.9185% | 0.3294 ms |
| Test B: Interaction holdout (S3) | 1.3750% | 1.0149% | 1.5636% | 1.6081% | 1.3398% | 1.3473% | 0.3205 ms |
Units: bulk modulus (GPa), Debye temperature (K), sound velocity (m/s), density (g/cm^3), thermal conductivity (W/(m*K)).
Strict reference validation plus out-of-family stress tests across 15 split scenarios.
| Property | Strict MAPE % (N) | Out-of-family weighted MAPE % (rows) |
|---|---|---|
| Crystal structure | 0.000 (59) | 0.000 (59) |
| Lattice constant (A) | 0.085 (59) | 0.086 (59) |
| Lattice constant (pm) | 0.085 (59) | 0.086 (59) |
| Atomic volume (A^3) | 0.497 (71) | 0.488 (71) |
| Band gap (eV) | 0.690 (274) | 0.985 (674) |
| Optical band gap (eV) | 0.527 (39) | 0.530 (39) |
| Dielectric constant | 0.910 (32) | 0.925 (32) |
| Refractive index | 0.658 (49) | 0.662 (49) |
| Reflectivity | 0.339 (28) | 0.338 (28) |
| Hardness (GPa) | 0.740 (44) | 0.724 (44) |
| Cv (J/mol*K) | 0.857 (39) | 0.858 (39) |
| Cv (J/kg*K) | 0.857 (39) | 0.858 (39) |
| Cp (J/mol*K) | 0.797 (81) | 0.801 (81) |
| Thermal expansion (1/K) | 0.903 (39) | 0.908 (39) |
| Melting point (K) | 0.947 (82) | 0.973 (82) |
| Density (g/cm^3) | 0.929 (162) | 0.965 (165) |
For crystal structure, values are mismatch-rate percentages (0% means all evaluated rows matched).
All models below are scored on the same strict splits. Lower MAPE is better.
| Model | Overall | Bulk Modulus | Debye Temp | Sound Velocity | Density | Thermal Cond. |
|---|---|---|---|---|---|---|
| FLUX | 1.1668% | 0.9056% | 1.3283% | 1.4405% | 1.2231% | 0.9185% |
| AFLOW adapter | 36.0693% | 14.3580% | 15.6790% | 29.5091% | 5.5695% | 122.9277% |
| JARVIS adapter | 10.9230% | 15.4191% | NA | NA | 6.3425% | NA |
| Matbench adapter | 18.4214% | 18.4214% | NA | NA | NA | NA |
| Model | Overall | Bulk Modulus | Debye Temp | Sound Velocity | Density | Thermal Cond. |
|---|---|---|---|---|---|---|
| FLUX | 1.3750% | 1.0149% | 1.5636% | 1.6081% | 1.3398% | 1.3473% |
| AFLOW adapter | 35.3566% | 14.5881% | 14.1047% | 30.0588% | 5.5493% | 119.6271% |
| JARVIS adapter | 10.9356% | 15.4816% | NA | NA | 6.2780% | NA |
| Matbench adapter | 18.4247% | 18.4247% | NA | NA | NA | NA |
Important: NA means that model did not provide that property in this comparison. Coverage differs by model and property.
Both benchmark tracks are frozen to public snapshots so every number can be rerun and verified.
Public export label uses reader-facing naming conventions only.
Band gap accuracy is reported on a dedicated page with its own dataset and metric definitions.
A curated showcase of defect-center and illumination behavior using the same deterministic universal runtime path.
| Case | Context | Expected | Predicted |
|---|---|---|---|
| Amethyst | Quartz + Fe center | Purple | Purple |
| Ruby | Corundum + Cr center | Red | Red |
| Emerald | Beryl + Cr/V center | Green | Green |
| Blue Topaz | Topaz + F-like center | Blue | Blue |
| Alexandrite (daylight) | Cr center, daylight profile | Green | Green |
| Alexandrite (incandescent) | Same material, warm-light profile | Red | Red |
Runtime remains deterministic and constant-time. No ML inference is used in final engine calculations.