Updated ERA Living Guidelines on the Responsible Use of Generative AI in Research

The European Commission has published a revised version of the ERA Living Guidelines on the Responsible Use of Generative AI in Research. The Guidelines have been updated to reflect the rapid developments in the field of generative AI and to take into account current technological changes and their impact on the scientific community.

The practical approach of the Guidelines, originally presented during the R&I Days 2024, has been maintained. At the same time, key principles of research integrity, including accountability, transparency and responsibility, have been adapted to the current use of generative AI in research. The Guidelines serve as a regulatory compass for research institutions, funding organisations, and researchers to ensure that these core principles are upheld in the context of generative AI systems over the long term.

Key Technical Updates and Additions

The update focuses primarily on shifting risk profiles during the daily use of AI infrastructures in scientific practice:

• Risk Assessments Regarding “Hidden Prompts”: Organizations and researchers are explicitly urged to consider the risks of concealed system instructions (hidden prompts). These refer to embedded commands within AI systems or documents that are not immediately visible to human users. They operate outside of human control and can lead to unnoticed manipulation of outputs. The Guidelines formulate specific mandates for this scenario:
  • When using generative AI as part of the internal processes, take into account the risks related to hidden prompts in content received by third parties, in particular in applications and proposals.
  • Research funding organisations remain aware that content may contain hidden prompts or invisible instructions intended to influence generative AI tools.
  • Where relevant, they set clear rules prohibiting such manipulation and introduce appropriate safeguards, in particular in the IT systems involved in the processing of that information.
• Interactions with Third-Party Tools: New, concrete recommendations have been integrated for handling external AI services. This particularly concerns the use of AI-driven tools in information management, as well as the automated transcription or summarization of meetings. The guidelines warn of the associated risks to confidentiality, data protection, and intellectual property rights. To address these challenges, the update requires organizations to:
  • Consider the use of generative AI by third parties that could be interacting with you/your organisation.
  • Researchers and organisations remain mindful that third parties may use generative AI tools. For example, in the context of meetings, note-taking, summarising discussions or creating overviews of documents (that could be confidential). Therefore, they can face similar risks to the ones described in this guidance, namely on confidentiality, data protection or intellectual property rights related to protected or original information.
  • Inform third parties about the use of these kinds of tools and the potential risks, for example by sharing this guidance.
  • In your contacts with third parties inform them if you are the one using those AI tools and how you are protecting collected information.

As generative AI is constantly evolving, these guidelines will be updated with regular feedback from the scientific community and stakeholders.

Sources:

A detailed overview of the changes is available via the official announcement on the European Commission's ERA News Portal.
The updated document and the announcement by the Directorate-General for Research and Innovation can be accessed through the European Commission's Research and Innovation Portal.