LettRAGraph at LLMs4OL 2026 Task Flagship: Prompt-Guided Ontology Extraction With Ensembling
DOI:
https://doi.org/10.52825/ocp.v11i.3655Keywords:
Ontology Learning, Knowledge Graph Extraction, Large Language Models, Prompt Engineering, Ensemble MethodsAbstract
We present a system for constructing ontology graphs from text using open-weight large language models. Our approach combines structured prompting, predicate normalization, and structural filtering. Prompt ablation experiments show that explicit taxonomy rules and closed-world constraints improve graph quality. We also introduce a conservative ensemble that retains only triples shared by two prompt configurations. On the official test set, the selected ensemble achieves Graph Similarity scores of 0.3584, 0.4872, and 0.5162 under exact, fuzzy, and semantic matching.
Downloads
References
[1] M. N. Asim, M. Wasim, M. U. G. Khan, W. Mahmood, and H. M. Abbasi, "A survey of ontology learning techniques and applications", Database, vol. 2018, p. bay101, Jan. 2018. ISSN: 1758-0463. DOI: 10.1093/database/bay101.
[2] P. Armary, C. B. El-Vaigh, O. Labbani Narsis, and C. Nicolle, "Ontology learning towards expressiveness: A survey", Comput. Sci. Rev., vol. 56, no. C, May 2025. ISSN: 1574-0137. DOI: 10.1016/j.cosrev.2024.100693.
[3] J. Plu, O. Moreno Escobar, E. Trouillez, A. Gapin, and R. Troncy, "A Comprehensive Benchmark for Evaluating LLM-Generated Ontologies", in 23rd Int. Semantic Web Conf. (ISWC), Special Session on Harmonising Generative AI and Semantic Web Technologies (HGAIS), CEUR, Ed., Baltimore, USA, 2024.
[4] P. Lisena, J. Plu, O. M. Escobar, E. Trouillez, and R. Troncy, "Pitfalls in AI-Generated Ontologies: Strategies for Detection and Mitigation", in LLMs4KGOE 2026, Int. Workshop on LLM-driven Knowledge Graph and Ontology Eng., co-located with ESWC 2026 (23rd European Semantic Web Conf.), CEUR-WS, Ed., Dubrovnik, Croatia, May 2026.
[5] J. Li, D. Garijo, and M. Poveda-Villalón, "Large Language Models for Ontology Engineering: A Systematic Literature Review", Semantic Web, vol. 17, no. 4, 2026. DOI: 10.1177/22104968261465514.
[6] L. Peng, P. Yang, Y. Juexiang, and L. Yuangan, "The construction and refined extraction techniques of knowledge graph based on large language models", Scientific Reports, vol. 16, no. 1, p. 8104, Feb. 2026. ISSN: 2045-2322. DOI: 10.1038/s41598-026-38066-w.
[7] H. Babaei Giglou, J. D'Souza, and S. Auer, "LLMs4OL: Large Language Models for Ontology Learning", in The Semantic Web – ISWC 2023, T. R. Payne, V. Presutti, G. Qi et al., Eds., Cham: Springer Nature Switzerland, 2023, pp. 408–427. ISBN: 978-3-031-47240-4.
[8] H. B. Giglou, S. Auer, and J. D'Souza, "LLMs4OL 2026 Overview: The 3rd Large Language Models for Ontology Learning Challenge", Open Conf. Proc., 2026.
[9] J. Plu, O. M. Escobar, E. Trouillez et al., "Text2KGBench-LettrIA: A refined benchmark for Text2Graph systems", in KBC-LM and LM-KBC Challenge (Knowledge Base Construction from Pre-trained Language Models) at ISWC 2025, 24th Int. Semantic Web Conf., 2-6 November 2025, Nara, Japan, CEUR-WS, Ed., Nara, Japan, Nov. 2025. [Online]. Available: https://ceur-ws.org/Vol-4041/paper3.pdf.
[10] T. Ehrhart, C. M. Songuon, J. Plu et al., "ELO-GRAG@EvalLLM 2026: Vers un système HybridRAG combinant graphes de connaissances et texte, évalué sur un corpus de défense en français", in Proc. of the EvalLLM 2026 Workshop: Atelier sur l'évaluation des modèles génératifs (LLM), le RAG et challenges, Nantes, France, Jun. 2026.
[11] Gemma Team, S. E. Abd, V. Aggarwal et al., "Gemma 4 Technical Report", Tech. Rep., 2026. [Online]. Available: 2607.02770.
[12] Qwen Team, "Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All", Apr. 2026. [Online]. Available: https://qwen.ai/blog?id=qwen3.6-35b-a3b.
[13] A. Kusupati, G. Bhatt, A. Rege et al., "Matryoshka representation learning", in Proc. of the 36th Int. Conf. on Neural Information Processing Systems, ser. NIPS '22, New Orleans, LA, USA: Curran Associates Inc., 2022. ISBN: 9781713871088.
Downloads
Published
How to Cite
Conference Proceedings Volume
Section
License
Copyright (c) 2026 Pasquale Lisena, Julien Plu, Oscar Moreno Escobar, Edouard Trouillez, Raphael Troncy

This work is licensed under a Creative Commons Attribution 4.0 International License.
Funding data
-
Bpifrance
Grant numbers DOS0256163/00