LettRAGraph at LLMs4OL 2026 Task Flagship: Prompt-Guided Ontology Extraction With Ensembling

Authors

DOI:

https://doi.org/10.52825/ocp.v11i.3655

Keywords:

Ontology Learning, Knowledge Graph Extraction, Large Language Models, Prompt Engineering, Ensemble Methods

Abstract

We present a system for constructing ontology graphs from text using open-weight large language models. Our approach combines structured prompting, predicate normalization, and structural filtering. Prompt ablation experiments show that explicit taxonomy rules and closed-world constraints improve graph quality. We also introduce a conservative ensemble that retains only triples shared by two prompt configurations. On the official test set, the selected ensemble achieves Graph Similarity scores of 0.3584, 0.4872, and 0.5162 under exact, fuzzy, and semantic matching.

Downloads

Download data is not yet available.

References

[1] M. N. Asim, M. Wasim, M. U. G. Khan, W. Mahmood, and H. M. Abbasi, "A survey of ontology learning techniques and applications", Database, vol. 2018, p. bay101, Jan. 2018. ISSN: 1758-0463. DOI: 10.1093/database/bay101.

[2] P. Armary, C. B. El-Vaigh, O. Labbani Narsis, and C. Nicolle, "Ontology learning towards expressiveness: A survey", Comput. Sci. Rev., vol. 56, no. C, May 2025. ISSN: 1574-0137. DOI: 10.1016/j.cosrev.2024.100693.

[3] J. Plu, O. Moreno Escobar, E. Trouillez, A. Gapin, and R. Troncy, "A Comprehensive Benchmark for Evaluating LLM-Generated Ontologies", in 23rd Int. Semantic Web Conf. (ISWC), Special Session on Harmonising Generative AI and Semantic Web Technologies (HGAIS), CEUR, Ed., Baltimore, USA, 2024.

[4] P. Lisena, J. Plu, O. M. Escobar, E. Trouillez, and R. Troncy, "Pitfalls in AI-Generated Ontologies: Strategies for Detection and Mitigation", in LLMs4KGOE 2026, Int. Workshop on LLM-driven Knowledge Graph and Ontology Eng., co-located with ESWC 2026 (23rd European Semantic Web Conf.), CEUR-WS, Ed., Dubrovnik, Croatia, May 2026.

[5] J. Li, D. Garijo, and M. Poveda-Villalón, "Large Language Models for Ontology Engineering: A Systematic Literature Review", Semantic Web, vol. 17, no. 4, 2026. DOI: 10.1177/22104968261465514.

[6] L. Peng, P. Yang, Y. Juexiang, and L. Yuangan, "The construction and refined extraction techniques of knowledge graph based on large language models", Scientific Reports, vol. 16, no. 1, p. 8104, Feb. 2026. ISSN: 2045-2322. DOI: 10.1038/s41598-026-38066-w.

[7] H. Babaei Giglou, J. D'Souza, and S. Auer, "LLMs4OL: Large Language Models for Ontology Learning", in The Semantic Web – ISWC 2023, T. R. Payne, V. Presutti, G. Qi et al., Eds., Cham: Springer Nature Switzerland, 2023, pp. 408–427. ISBN: 978-3-031-47240-4.

[8] H. B. Giglou, S. Auer, and J. D'Souza, "LLMs4OL 2026 Overview: The 3rd Large Language Models for Ontology Learning Challenge", Open Conf. Proc., 2026.

[9] J. Plu, O. M. Escobar, E. Trouillez et al., "Text2KGBench-LettrIA: A refined benchmark for Text2Graph systems", in KBC-LM and LM-KBC Challenge (Knowledge Base Construction from Pre-trained Language Models) at ISWC 2025, 24th Int. Semantic Web Conf., 2-6 November 2025, Nara, Japan, CEUR-WS, Ed., Nara, Japan, Nov. 2025. [Online]. Available: https://ceur-ws.org/Vol-4041/paper3.pdf.

[10] T. Ehrhart, C. M. Songuon, J. Plu et al., "ELO-GRAG@EvalLLM 2026: Vers un système HybridRAG combinant graphes de connaissances et texte, évalué sur un corpus de défense en français", in Proc. of the EvalLLM 2026 Workshop: Atelier sur l'évaluation des modèles génératifs (LLM), le RAG et challenges, Nantes, France, Jun. 2026.

[11] Gemma Team, S. E. Abd, V. Aggarwal et al., "Gemma 4 Technical Report", Tech. Rep., 2026. [Online]. Available: 2607.02770.

[12] Qwen Team, "Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All", Apr. 2026. [Online]. Available: https://qwen.ai/blog?id=qwen3.6-35b-a3b.

[13] A. Kusupati, G. Bhatt, A. Rege et al., "Matryoshka representation learning", in Proc. of the 36th Int. Conf. on Neural Information Processing Systems, ser. NIPS '22, New Orleans, LA, USA: Curran Associates Inc., 2022. ISBN: 9781713871088.

Downloads

Published

2026-11-03

How to Cite

Lisena, P., Plu, J., Escobar, O. M., Trouillez, E., & Troncy, R. (2026). LettRAGraph at LLMs4OL 2026 Task Flagship: Prompt-Guided Ontology Extraction With Ensembling. Open Conference Proceedings, 11. https://doi.org/10.52825/ocp.v11i.3655

Conference Proceedings Volume

Section

LLMs4OL 2026 Task Participant Short Papers

Funding data