As part of the AI4SEA project, over the past few months, we have made significant progress in the 3D semantic modeling of underwater environments using the Stonefish simulator. We have adapted the ConceptGraphs architecture—originally designed for indoor environments—to our underwater scenarios featuring Posidonia seagrass meadows, Autonomous Underwater Vehicles (AUVs), platforms, Posirocks, and other elements.
Building on 3D point cloud reconstructions and computer vision models (YOLO and SAM), a Large Language Model (LLM) automatically generates semantic labels and bounding boxes to identify key elements of the seabed.
We are currently optimizing the system to address challenges related to domain shift and reconstruction noise, laying the groundwork for autonomous semantic understanding of the complex underwater environments that AI4SEA will need to navigate and operate in.
