Author(s): Enya Roseli Enriquez Brambila; Gerlad Corzo; Michael Mcclain; Dimitri Solomatine
Linked Author(s): Dimitri Solomatine
Keywords: Multimodal AI CLIP ZeroShot ChatGPT Ground Based Images River Health
Abstract: Assessing river health is essential for sustainable water management, but traditional methods like the Riparian Quality Index (RQI) require expert-led, resource-intensive fieldwork, limiting scalability. This study explores using ChatGPT-4o, a multimodal AI model with CLIP zero-shot learning, to assess river health through RQI-guided photographic analysis. Using 21 photos from the El Coajinque catchment in Jalisco, Mexico, RQI indicators were scored based on structured textual protocols embedded into ChatGPT-4o. The AI was tasked with interpreting photographic data to assign scores to ecological indicators. Two tests were conducted: Test A analyzed single river landscape photos, while Test B used multi-photo sets capturing landscape, vegetation, and bank conditions. Test B achieved 85.7% accuracy in replicating expert assessments, outperforming Test A's 57.1%. This proof of concept demonstrates the integration of structured textual rules into a multimodal AI framework, achieving high accuracy despite challenges with ambiguous indicators. While the methodology was manually developed, ongoing experiments aim to automate processes and enhance reproducibility by optimizing parameters like temperature settings. This approach reduces errors and ensures stability, paving the way for scalable, reliable AI tools in river health assessment and sustainable resource management.
Year: 2025