DREAM-LLMs at LLMs4OL 2025 Task B: A Deliberation-Based Reasoning Ensemble Approach With Multiple Large Language Models for Term Typing in Low-Resource Domains

Abstract

The LLMs4OL Challenge at ISWC 2025 aims to advance the integration of Large Language Models (LLMs) and Ontology Learning (OL) across four key tasks: (1) Text2Onto, (2) Term Typing, (3) Taxonomy Discovery, and (4) Non-Taxonomic Relation Extraction. Our work focuses on the Term Typing Prediction task, where prompting LLMs has shown strong potential. However, in low-resource domains, relying on a single LLM is often insufficient due to domain-specific knowledge gaps and limited exposure to specialized terminology, which can lead to inconsistent and biased predictions. To address this challenge, we propose DREAM-LLMs: a Deliberation-based Reasoning Ensemble Approach with Multiple Large Language Models. Our method begins by crafting few-shot prompts using training examples and querying four advanced LLMs independently: ChatGPT-4o, Claude Sonnet 4, DeepSeek-V3, and Gemini 2.5 Pro. Each model outputs a predicted label along with a brief justification. To reduce model-specific bias, we introduce a deliberation step, in which one LLM reviews the predictions and justifications from the other three to produce a final decision. We evaluate DREAM-LLMs on three low-resource domain datasets: OBI, MatOnto, and SWEET using F1-score as the evaluation metric. The results, 0.908 for OBI, 0.568 for MatOnto, and 0.593 for SWEET, demonstrate that our ensemble strategy significantly improves performance, highlighting the promise of collaborative LLM reasoning in low-resource environments.