2025 · ACL 2025 — KnowFM: Towards Knowledgeable Foundation Models workshop

Stress-Testing Multimodal Foundation Models for Crystallographic Reasoning

Can Polat , Hasan Kurban , Erchin Serpedin , Mustafa Kurban

Research highlights

Testing crystallographic reasoning beyond familiar examples

Evaluate foundation models on crystal geometry, composition shifts, and physical consistency.

Why we investigated it

Plausible structural descriptions can contain incorrect lattice parameters or impossible geometric relationships.

What the work contributes

Spatial- and compositional-exclusion protocols test vision-language models using geometric error, consistency, and hallucination measures.

What the evidence shows

The study provides physically grounded diagnostics across nine models, separating numerical errors from invalid structural claims.

Build on this work

Use the evaluation protocols to test crystallographic reasoning under held-out sizes and compositions.

  • Crystallographic reasoning
  • Vision-language models
  • Hallucination
  • Physical consistency
Cite
@article{polat2025stresstesting,
  title = {Stress-Testing Multimodal Foundation Models for Crystallographic Reasoning},
  author = {Can Polat and Hasan Kurban and Erchin Serpedin and Mustafa Kurban},
  year = 2025,
  journal = {ACL 2025 — KnowFM: Towards Knowledgeable Foundation Models workshop},
  doi = {10.18653/v1/2025.knowllm-1.5},
}