← Benchmarks | Solvation

SOTA-Level Explicit Solvation Benchmark OFFICIAL PACKET

Production explicit-solvation benchmark packet with SOTA-level FreeSolv hydration accuracy, curated non-water external packets, a 436-case large-scale non-water packet, and strict repeated-holdout validation for native non-water explicit carriers.

SOTA-level

FreeSolv Hydration Accuracy

0.3295

MAE (kcal/mol)

1.3787

MAE (kJ/mol)

642

FreeSolv Cases

0.9857

R squared

Hydration Accuracy (FreeSolv)

SOTA-level FreeSolv hydration accuracy from the official explicit-solvation packet.

Dataset	Cases	MAE (kcal/mol)	RMSE (kcal/mol)	MAE (kJ/mol)	RMSE (kJ/mol)	Pearson r	R squared
FreeSolv screening subset	642	0.3295	0.4593	1.3787	1.9219	0.9928	0.9857

Auxiliary metrics: max absolute error 2.4019 kcal/mol, median absolute error 0.2427 kcal/mol, within-uncertainty fraction 0.7492.

Solvent Model Coverage Status

Explicit carrier status in production benchmark policy.

Model	Solvent	Status	External benchmark in this packet
Water (TIP3P)	Water	Production	Yes
Water (TIP4P)	Water	Production	Reference policy-listed model
Methanol	Methanol	Native explicit carrier	External phase-1 packet + strict holdout
Ethanol	Ethanol	Native explicit carrier	External phase-1 packet + strict holdout
Acetonitrile	Acetonitrile	Native explicit carrier	External phase-1 packet + strict holdout
DMSO	DMSO	Native explicit carrier	External phase-1 packet + strict holdout

Non-Water External Phase-1 Packets

Curated external screening datasets for the native methanol, ethanol, acetonitrile, and DMSO carriers.

194

Curated External Cases

0.1428-0.1823

MAE Range (kcal/mol)

Published Solvent Packets

0.1639

Mean Solvent MAE (kcal/mol)

Solvent	Cases	MAE (kcal/mol)	RMSE (kcal/mol)	R squared	Worst Case Abs Error
Methanol	97	0.1636	0.2249	0.9880	0.7772
Ethanol	65	0.1428	0.1919	0.9944	0.6431
Acetonitrile	15	0.1670	0.2224	0.9372	0.5027
DMSO	17	0.1823	0.2204	0.9712	0.3796

These are curated external phase-1 solvent packets. They expand public non-water coverage but are smaller than the water hydration corpus.

Non-Water Large-Scale External Packet

Broader SolProp-enriched external validation across methanol, ethanol, acetonitrile, and DMSO.

436

Total External Cases

0.6155

Weighted MAE (kcal/mol)

0.3103-0.8817

Solvent MAE Range

Phase-1 Total Gap Remaining

Solvent	Cases	MAE (kcal/mol)	RMSE (kcal/mol)	R squared	Max Abs Error
Methanol	161	0.5308	1.1077	0.9543	7.0171
Ethanol	147	0.8817	1.6501	0.9273	6.7027
Acetonitrile	67	0.3103	0.4026	0.9462	0.9455
DMSO	61	0.5330	0.7708	0.7379	2.6925

This packet is broader and harder than the curated 194-case subset. It is published separately and should not be conflated with the smaller curated external packet.

Native Non-Water Carrier Holdout

Strict repeated holdout validation on the expanded 436-case SolProp-enriched non-water corpus.

0.5422

Overall MAE (kcal/mol)

436

Native-carrier cases

Repeated holdout splits

≤ 1.0081

Worst solvent/category MAE

Solvent	Cases	Split MAE (kcal/mol)	Std Dev	Worst Category	Worst Category MAE
Methanol	161	0.4686	0.0615	amide_primary_secondary	0.5764
Ethanol	147	0.7955	0.1179	phenolic_polar_aromatic	1.0081
Acetonitrile	67	0.2814	0.0794	general	0.3191
DMSO	61	0.3741	0.0973	heavy_halomethane	0.4546

Strict repeated holdout remains the main robustness check for the native non-water carriers on the expanded corpus; curated and large-scale external packets are published alongside it as separate views.

Failure Analysis Highlights

Largest remaining error bands used for calibration targeting.

Highest-MAE chemotypes (n >= 10)

Chemotype	n	MAE (kcal/mol)
halophenol_like	11	0.5812
chlorinated_hydrophobe	22	0.4713
phenolic_polar_aromatic	30	0.4178
monohaloalkane_long	17	0.4095
alkoxy_rich	11	0.4017
hydrophobe	36	0.3756

Top absolute-error cases

Case ID	Chemotype	Abs Error (kcal/mol)
mobley_5732611	general	2.4019
mobley_3325209	general	2.4012
mobley_2763835	hydrophobe	1.6854
mobley_2751110	phenolic_polar_aromatic	1.6367
mobley_1178614	halophenol_like	1.5996

Scope and contract

What is benchmarked

Water hydration free energy on external FreeSolv references; this is the SOTA-level accuracy claim.

What this is not

A claim of full broad-corpus parity for every non-water solvent. Non-water coverage is published here as curated external packets, a broader SolProp-enriched external packet, and strict repeated holdout.

Execution profile

Public screening run with fast hydration evaluation settings.

Reproducibility

Full summary JSON, case table CSV/JSON, failure note, and methodology are published.

Download benchmark package

Machine-readable and human-readable artifacts for independent review.

Originator: FluxMateria

Benchmark summary JSON

Headline metrics, chemotype breakdown, and outlier cases.

Download JSON

Case-level benchmark CSV

All 642 cases with predicted/reference values and signed errors.

Download CSV

Case-level benchmark JSON

Structured case table for programmatic ingestion.

Download JSON

Methodology note

Benchmark scope, scoring policy, and run configuration.

Download MD

Error breakdown note

Chemotype residual analysis and calibration targets.

Download MD

Non-water holdout summary JSON

Sanitized strict repeated-holdout summary for the native non-water carrier models.

Download JSON

Non-water holdout category CSV

Per-solvent chemotype MAE table for the strict non-water holdout snapshot.

Download CSV

Non-water holdout methodology note

Public methodology summary for the native-carrier strict holdout protocol.

Download MD

Non-water external large-scale summary JSON

Sanitized combined summary for the broader SolProp-enriched non-water external packet.

Download JSON

Non-water external large-scale methodology note

Public methodology summary for the broader SolProp-enriched non-water validation packet.

Download MD

Non-water external large-scale packet manifest

Combined manifest and solvent table for the 436-case large-scale non-water packet.

Download JSON

Non-water external packet summary JSON

Sanitized combined summary for the methanol, ethanol, acetonitrile, and DMSO external phase-1 packets.

Download JSON

Non-water external methodology note

Public methodology summary for the curated external non-water solvent packets.

Download MD

Non-water external packet manifest

Combined packet manifest with artifact hashes for the four external non-water solvent sets.

JSON MD

Packet manifest (JSON + Markdown)

Snapshot metadata, artifact hashes, and benchmark contract.

JSON MD

Need module-level context?

This page is the explicit-solvation benchmark packet. For product context and broader solvation capability coverage, use the module page.

Solvation Module Request Access

Benchmark basis

This benchmark reports hydration and non-water solvation results together. The aggregate page is mixed because solvent models and validation packets are reported separately.

Mixed basis