Trustworthy Scientific Narrative Generation Through Computational Provenance and Dynamic Authoring Frameworks

Augustus Ellerm; Benjamin Adams; Mark Gahegan

doi:10.52825/ocp.v8i.3174

Authors

Augustus Ellerm University of Auckland
Benjamin Adams University of Auckland https://orcid.org/0000-0002-1657-9809
Mark Gahegan University of Auckland

DOI:

https://doi.org/10.52825/ocp.v8i.3174

Keywords:

Scholarly Publishing, Large Language Models, Dynamic Authoring Framework, Provenance

Abstract

Despite the growing quantity of born-digital research, scholarly articles remain tethered to flattened PDFs -- unable to expose or verify the computations they summarize. We present a publication container that couples provenance-generating eScience infrastructures with Dynamic Authoring Frameworks (DAFs). Using this method, an article is modelled as a set of symbolic operations over provenance. The result is a procedural narrative whose claims are anchored to deterministic combinations of operations and provenance, enabling verification. To enhance this method we place LLM inference tasks inside the DAF for small, constrained tasks. This mitigates hallucination, supports granular attribution, and enables researchers to move "warrant" making away from LLM inference, to the explicit operations within the DAF. Using a remote-sensing case study (CoastSat), we show how this method can produce consistent, accurate, and generative methodological descriptions from provenance. We argue that for modern scholarly communication to support generative text, it must move beyond "flat" scholarly articles towards more formal representations of authorship.

Downloads

Download data is not yet available.

References

[1] S. Harnad, "Post-Gutenberg Galaxy: The Fourth Revolution in the Means of Production of Knowledge", The Public-Access Computer Systems Review, vol. 2, no. 1, pp. 39–53, 1991.

[2] P. E. Bourne, T. W. Clark, R. Dale et al., "Improving the Future of Research Communications and E-Scholarship (Dagstuhl Perspectives Workshop 11331)", Dagstuhl Manifestos, vol. 1, no. 1, pp. 41–60, 2012. DOI: 10.4230/DagMan.1.1.41.

[3] D. Shotton, "Semantic Publishing: The Coming Revolution in Scientific Journal Publishing", Learn. Publ., vol. 22, no. 2, pp. 85–94, 2009. DOI: 10.1087/2009202.

[4] T. Clark, P. N. Ciccarese, and C. A. Goble, "Micropublications: A Semantic Model for Claims, Evidence, Arguments and Annotations in Biomedical Communications", J. Biomed. Semantics, vol. 5, p. 28, 2014. DOI: 10.1186/2041-1480-5-28.

[5] C. Tenopir, D. W. King, L. Christian, and R. Volentine, "Scholarly Article Seeking, Reading, and Use: A Continuing Evolution from Print to Electronic in the Sciences and Social Sciences", Learn. Publ., vol. 28, no. 2, pp. 93–105, 2015. DOI: 10.1087/20150203.

[6] M. Baker, "1,500 Scientists Lift the Lid on Reproducibility", Nature, vol. 533, no. 7604, pp. 452–454, 2016. DOI: 10.1038/533452a.

[7] M. D. Wilkinson, M. Dumontier, I. J. Aalbersberg et al., "The FAIR Guiding Principles for Scientific Data Management and Stewardship", Scientific Data, vol. 3, no. 1, p. 160018, 2016. DOI: 10.1038/sdata.2016.18.

[8] F. M. Howard, A. Li, M. F. Riffon, E. Garrett-Mayer, and A. T. Pearson, "Characterizing the Increase in Artificial Intelligence Content Detection in Oncology Scientific Abstracts from 2021 to 2023", JCO Clin. Cancer Inform., vol. 8, no. 8, p. e2400077, 2024. DOI: 10.1200/CCI.24.00077.

[9] Nature, "Tools Such as ChatGPT Threaten Transparent Science; Here Are Our Ground Rules for Their Use", Nature, vol. 613, no. 7945, p. 612, 2023. DOI: 10.1038/d41586-023-00191-1.

[10] International Committee of Medical Journal Editors, "Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals", Chin. Nurs. Res., vol. 2, no. 4, pp. I–XIII, 2015.

[11] C. Zielinski, M. Winker, R. Aggarwal et al., "WAME Recommendations on ChatGPT and Chatbots in Relation to Scholarly Publications", Natl. Med. J. India, vol. 36, no. 1, pp. 1–4, 2023. DOI: 10.25259/NMJI_365_23.

[12] L. Huang, W. Yu, W. Ma et al., "A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions", ACM Trans. Inf. Syst., vol. 43, no. 2, pp. 46:1–46:55, 2025. DOI: 10.1145/3703155.

[13] X. Wei, N. Kumar, and H. Zhang, "Addressing Bias in Generative AI: Challenges and Research Opportunities in Information Management", Information & Management, vol. 62, no. 2, p. 104103, 2025. DOI: 10.1016/j.im.2025.104103.

[14] B. Huang, C. Chen, and K. Shu, "Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges", ACM SIGKDD Explorations Newsletter, vol. 26, no. 2, pp. 21–43, 2024. DOI: 10.1145/3715073.3715076.

[15] Y. Song, G. Wang, S. Li, and B. Y. Lin, "The Good, the Bad, and the Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism", in Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Albuquerque, NM, USA: Association for Computational Linguistics, 2025, pp. 4195–4206. DOI: 10.18653/v1/2025.naacl-long.211.

[16] N. Wilkins-Diehr, "Special Issue: Science Gateways—Common Community Interfaces to Grid Resources", Concurr. Comput., vol. 19, no. 6, pp. 743–749, 2007. DOI: 10.1002/cpe.1098.

[17] L. Candela, D. Castelli, and P. Pagano, "Virtual Research Environments: An Overview and a Research Agenda", Data Sci. J., vol. 12, pp. GRDI75–GRDI81, 2013. DOI: 10.2481/dsj.GRDI-013.

[18] D. E. Knuth, "Literate Programming", Comput. J., vol. 27, no. 2, pp. 97–111, 1984. DOI: 10.1093/comjnl/27.2.97.

[19] M. Alser, B. Lawlor, R. J. Abdill et al., "Packaging and Containerization of Computational Methods", Nat. Protoc., vol. 19, no. 9, pp. 2529–2539, 2024. DOI: 10.1038/s41596-024-00986-0.

[20] P. Groth, and L. Moreau, "PROV-Overview: An Overview of the PROV Family of Documents", World Wide Web Consortium, Tech. Rep., 2013, W3C Note 30 April 2013.

[21] C. Goble, S. Cohen-Boulakia, S. Soiland-Reyes et al., "FAIR Computational Workflows", Data Intelligence, vol. 2, no. 1-2, pp. 108–121, 2020. DOI: 10.1162/dint_a_00033.

[22] S. B. Davidson, and J. Freire, "Provenance and Scientific Workflows: Challenges and Opportunities", in Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, Vancouver, BC, Canada: ACM, 2008, pp. 1345–1350. DOI: 10.1145/1376616.1376772.

[23] P. Di Tommaso, M. Chatzou, E. W. Floden, P. P. Barja, E. Palumbo, and C. Notredame, "Nextflow Enables Reproducible Computational Workflows", Nat. Biotechnol., vol. 35, no. 4, pp. 316–319, 2017. DOI: 10.1038/nbt.3820.

[24] J. Vivian, A. A. Rao, F. A. Nothaft et al., "Toil Enables Reproducible, Open Source, Big Biomedical Data Analyses", Nat. Biotechnol., vol. 35, no. 4, pp. 314–316, 2017. DOI: 10.1038/nbt.3772.

[25] Arvados, "Arvados Homepage", https://arvados.org/.

[26] S. Leo, M. R. Crusoe, L. Rodr'iguez-Navas et al., "Recording Provenance of Workflow Runs with RO-Crate", PLoS One, vol. 19, no. 9, p. e0309210, 2024. DOI: 10.1371/journal.pone.0309210.

[27] F. Z. Khan, S. Soiland-Reyes, M. R. Crusoe, A. Lonie, and R. Sinnott, "CWLProv: Interoperable Retrospective Provenance Capture and Its Challenges", in Proceedings of the 7th International Provenance and Annotation Workshop (IPAW 2018), Springer, 2018. DOI: 10.5281/zenodo.1208478.

[28] L. Flower, and J. R. Hayes, "A Cognitive Process Theory of Writing", Coll. Compos. Commun., vol. 32, no. 4, pp. 365–387, 1981. DOI: 10.2307/356600.

[29] S. P. Witte, and L. Faigley, "Coherence, Cohesion, and Writing Quality", Coll. Compos. Commun., vol. 32, no. 2, pp. 189–204, 1981. DOI: 10.2307/356693.

[30] L. Groarke, "Informal Logic", in The Stanford Encyclopedia of Philosophy, E. N. Zalta, Ed., Fall 2021 Edition, Metaphysics Research Lab, Stanford University, 2021.

[31] M. Aufreiter, A. Pawlik, and N. Bentley, "Stencila – an Office Suite for Reproducible Research", https://elifesciences.org/labs/c496b8bb/stencila-an-office-suite-for-reproducible-research, elifesciences.org, 2018.

[32] R. M. Langridge, J. Rowland, P. Villamor et al., "Coseismic Rupture and Preliminary Slip Estimates for the Papatea Fault and Its Role in the 2016 MtextsubscriptW 7.8 Kaik=oura, New Zealand, Earthquake", Bull. Seismol. Soc. Am., vol. 108, no. 3B, pp. 1596–1622, 2018. DOI: 10.1785/0120170336.

[33] Z. Xu, S. Jain, and M. Kankanhalli, "Hallucination Is Inevitable: An Innate Limitation of Large Language Models", arXiv [cs.CL], 2024. [Online]. Available: 2401.11817.

[34] N. F. Liu, K. Lin, J. Hewitt et al., "Lost in the Middle: How Language Models Use Long Contexts", Trans. Assoc. Comput. Linguist., vol. 12, pp. 157–173, 2024. DOI: 10.1162/tacl_a_00638.

[35] H. Jiang, Q. Wu, C. Lin, Y. Yang, and L. Qiu, "LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models", in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, H. Bouamor, J. Pino, and K. Bali, Eds., Singapore: Association for Computational Linguistics, 2023, pp. 13358–13376. DOI: 10.18653/v1/2023.emnlp-main.825.

[36] Z. Ji, N. Lee, R. Frieske et al., "Survey of Hallucination in Natural Language Generation", ACM Comput. Surv., vol. 55, no. 12, 2023. ISSN: 0360-0300. DOI: 10.1145/3571730.

[37] A. Ellerm, M. Gahegan, and B. Adams, "LivePublication: The Science Workflow Creates and Updates the Publication", in 2023 IEEE 19th International Conference on e-Science (e-Science), IEEE, 2023, pp. 1–10. DOI: 10.1109/e-Science58273.2023.10254857.

Trustworthy Scientific Narrative Generation Through Computational Provenance and Dynamic Authoring Frameworks

Authors

DOI:

Keywords:

Abstract

Downloads

References

Downloads

Published

How to Cite

Conference Proceedings Volume

Section

License

Funding data