2025 · Machine Learning Science and Technology

CrysMTM: a multiphase, temperature-resolved, multimodal dataset for crystalline materials

Can Polat , Erchin Serpedin , Mustafa Kurban , Hasan Kurban

Research highlights

CrysMTM: testing models under thermal shift

Benchmark materials models across temperature, crystal phase, and input modality.

Why we investigated it

Models accurate near their training conditions may fail when thermal changes alter geometry and properties.

What the work contributes

CrysMTM pairs coordinates, images, and text for about 30,000 TiO2 samples and benchmarks 18 models.

What the evidence shows

The evaluated GNNs show roughly threefold higher error under thermal shift. A few-shot multimodal model narrows the reported in-distribution and out-of-distribution gap.

Build on this work

Use the benchmark's phase and temperature controls to test model robustness beyond random train-test splits.

  • Thermal generalization
  • Multimodal benchmarks
  • TiO2
  • Distribution shift
Abstract
Abstract We present CrysMTM, a large-scale, multimodal dataset designed to benchmark temperature- and phase-sensitive machine learning models for crystalline materials. The dataset comprises approximately 30 000 atomistic samples covering the three primary polymorphs of titanium dioxide–anatase, brookite, and rutile–each evaluated across a temperature spectrum ranging from cryogenic to ambient and elevated conditions. Each data entry integrates three complementary modalities: (1) three-dimensional atomic coordinates, (2) RGBA molecular visualizations, and (3) structured textual metadata encompassing geometric descriptors, local bonding environments, and phase transformation parameters. This multimodal structure enables both supervised and self-supervised learning across graph-based, image-based, and language-based architectures. CrysMTM supports rigorous evaluation of model robustness under thermal perturbations and crystallographic phase transitions. Baseline benchmarking across 18 models–including graph neural networks (GNNs), convolutional neural networks, and foundation models–reveals significant property-specific challenges. For example, bandgap predictions exhibit errors exceeding 25%, while volumetric expansion and atomic displacement estimations frequently deviate by more than 100%. Even state-of-the-art GNNs, which achieve an average in-distribution (ID) mean absolute percentage error of approximately 20%, show a threefold increase under out-of-distribution (OOD) thermal conditions. In contrast, a few-shot multimodal large language model reduces global prediction error from 96% to 23% and narrows the performance gap between ID and OOD cases to just four percentage points. These results highlight both the selective difficulty posed by temperature-sensitive geometric targets and the considerable room for innovation in model design. All dataset files, model implementations, and pretrained checkpoints are available at https://github.com/KurbanIntelligenceLab/CrysMTM .
Cite
@article{polat2025crysmtm,
  title = {CrysMTM: a multiphase, temperature-resolved, multimodal dataset for crystalline materials},
  author = {Can Polat and Erchin Serpedin and Mustafa Kurban and Hasan Kurban},
  year = 2025,
  journal = {Machine Learning Science and Technology},
  doi = {10.1088/2632-2153/adf9bc},
}