Since its founding, ROR has been committed to providing open infrastructure for the global community. As part of this commitment, in December 2020, ROR released a public self-assessment of our alignment with the Principles of Open Scholarly Infrastructure (POSI).

At the time, the POSI framework, originally articulated in 2015, had been recently (re)codified as a set of concrete guidelines, inspiring a small group of open scholarly infrastructure initiatives to assess how they were aligned with these principles.

For ROR, this context was particularly relevant as it was in the early stages of formalizing an operating model and exploring pathways to long-term sustainability. POSI provided a helpful roadmap to inform ROR’s strategic decision-making and a useful framework for contextualizing and organizing its operations.

A revised version of the Principles of Open Scholarly Infrastructure (v2.0) was released in October 2025 (following a minor v1.1 revision in November 2023). As an early POSI adopter and a member of the POSI adopters group that formed after the 2020 recodification, ROR was involved in shaping the revision alongside other community collaborators. The v2.0 revision does not alter the spirit or intent of the principles, but rather clarifies and expands on some key points and nuances that emerged through community consultations and also in response to ongoing changes in the scholarly infrastructure landscape.

In the wake of the POSI v2.0 release, in recognition of the five years since our initial self-assessment, and having recently marked the milestone of ROR’s seventh anniversary, we wanted to take this opportunity to reflect on the current state of ROR’s alignment with the POSI framework and re-assert our commitment to providing open scholarly infrastructure for the global community.

Evaluation process

The POSI framework focuses on three key areas: accountability (governance), resourcing (sustainability), and protection of community interests (insurance). Twenty principles are articulated within these three areas. The principles are intended to provide guidance; they are not intended to function as a set of rules or a checklist, and it is not expected or required that all of the principles will be relevant for every organization or initiative.

For ROR’s revised self-assessment, we reviewed the updates from POSI version 1.1 and version 2.0 and evaluated ROR’s current and planned activities against these newer versions of the principles.

Similar to the 2020 assessment, we evaluated degrees of alignment with the principles using a “traffic light” ranking, where green indicates strong alignment, yellow indicates partial alignment or alignment in process, and red indicates weak alignment or non-alignment. There is no standard POSI assessment metric; some organizations have adopted the traffic light framework while others have taken different approaches. We present the results of the new assessment alongside the results from the 2020 assessment to indicate where alignment has remained consistent or evolved.

As ROR is a jointly operated initiative and not a standalone organization by design, some of the principles are less applicable than others. If an item is marked as green, this doesn’t necessarily mean ROR does this perfectly or that our work in this area is done; it simply indicates that we feel confident about our level of alignment and continued ability to maintain this alignment.

While we believe that the assessment indicators are helpful for understanding the state of ROR’s alignment at a glance, we also take seriously the details behind the indicators, and care about documenting and understanding the specific actions and decisions make alignment possible, so that we can integrate this work as a daily practice and continue to seek opportunities for growth and improvements over time.

Governance

ROR is a global registry that aims to represent all research organizations in the world and to be adopted and used all over the world. ROR IDs and accompanying metadata records encompass a diverse and global set of research organizations and organization types. The registry continues to be updated to include new organizations, particularly from regions historically underrepresented in research infrastructure, and to reflect metadata updates that support global adoption, such as including name variants in multiple languages and character sets.

ROR infrastructure is freely and openly available and can be used in any type of system and anywhere in the world without restrictions.

ROR community activities aim to reflect and support global usage and engagement. Community meetings and case studies feature a range of users and implementations. ROR’s Steering Group, Curation Advisory Board, and community forums similarly encompass and reflect diverse national and regional perspectives as well as a mix of organization types.

While ROR communications and community activities are primarily conducted in English and meetings are primarily conducted during Western Hemisphere working hours, we aim to broaden the linguistic and geographic scope of our communications and activities by producing translations of key community resources, using video translations and captions, scheduling some meetings and events in alternate time zones, and conducting meetings and communications in non-English languages when possible.

ROR is not a standalone organization by design and therefore has no official mechanism for board governance. ROR governance operations are carried out by three nonprofit organizations that are representative of ROR’s key stakeholder communities and specified in an official Memorandum of Agreement.

ROR also engages stakeholder input and guidance on strategic directions through its Steering Group, open community calls, Curation Advisory Board, and public calls for feedback.

As a multi-organization initiative, ROR’s governance model is implemented by its three operating organizations, which are nonprofit organizations that collectively represent key stakeholder communities for ROR and whose commitment to ROR has been approved by their respective governing bodies.

ROR is designed to be a community-driven open infrastructure initiative that has no membership or service model by design and that encourages active community engagement to inform its ongoing activities. ROR infrastructure can be accessed and used by anyone without a membership or a license or any other type of restriction. ROR community activities, such as community meetings and events, calls for feedback, bug reports and feature suggestions, and curation requests are openly available for anyone to opt-in.

As described in more detail elsewhere in this assessment, while ROR’s community activities and communications are conducted primarily in English during Western Hemisphere working hours, we take steps to support multilingual, global, multi-modal, and asynchronous engagement, to convey that all community members are welcome to participate regardless of where they work or how they prefer to communicate. We also seek to highlight users and adopters from a range of organization types and industries.

The registry’s inclusion criteria for organizations is based on adherence to ROR’s scope, which is established by community experts and stakeholders. Any organization can be included in the registry within this scope; there are no fees or licenses or other permissions required to add an organization to the registry or to maintain the metadata associated with the organization’s record.

ROR convenes a Curation Advisory Board and Steering Group made up of members of the community that are invited to provide specific guidance and advice on ROR activities. These groups do not have formal governance authority over ROR. Members are currently selected through an invitation process that is overseen by ROR’s operating organizations and core staff members; anyone is welcome to express interest in participating. Group operations are documented on GitHub and the ROR website.

ROR is operated as a collaborative initiative by California Digital Library, Crossref, and DataCite. The three ROR governing organizations collectively assume and share responsibility for ROR governance, operations, resourcing, and decision-making. These responsibilities are defined in a Memorandum of Agreement (MOA) that is approved by the governance bodies of each organization, renewed on a regular basis, and published on the ROR website. The MOA outlines how ROR governing operations are conducted and also stipulates specific principles of governance, for example, that ROR cannot be governed by, purchased by, controlled by, sold to, or otherwise transferred to a commercial entity.

ROR’s Memorandum of Agreement was first signed in 2020. Since then, the ROR operating organizations have renewed and revised the agreement multiple times to provide more clarity and transparency about ROR’s governance model and to keep the agreement aligned with current operations. The Memorandum of Agreement is now also made available on the ROR website.

ROR’s primary focus is to provide useful infrastructure for the community in the form of data and services that support a variety of use cases and needs. ROR does not lobby nor include regulatory change as part of its remit.

ROR operates on a core set of values and principles, including but not limited to POSI, based on the premise that scholarly infrastructure should be open. These values and principles are not motivated by self-interest but by the belief that open infrastructure provides a collective benefit.

ROR team members and community stakeholders participate in numerous activities and initiatives to advance ROR’s objectives in the name of the common good. Some of these activities and initiatives may directly or indirectly play a role in influencing policy decisions, such as in the emergence of national persistent identifier policies in multiple countries that may specifically mention ROR, but that are focused on addressing the needs of the global research community.

ROR operations and governance structures have accounted for long-term stewardship as well as future scenarios in which the service might be wound down.

ROR data is openly available under a CC0 waiver by default, allowing access to persist beyond the life of the initiative. ROR code is openly licensed on GitHub and can be forked by anyone. ROR data files are published on Zenodo and archived through Zenodo’s processes.

ROR’s operating organizations have a Memorandum of Agreement, which is published on the ROR website, that describes how ROR would be shut down and how relevant responsibilities could be transferred to responsible successors, if applicable. The succession plan also accounts for assets such as code repositories, cash funds, and technical infrastructure.

ROR’s operating organization and core team members regularly assess ROR’s strategic objectives, level of adoption and usage, and ongoing progress in order to ensure that ROR’s offering is aligned with community needs.

In the event that ROR’s operating organizations identify evidence that ROR is no longer needed, they will take the necessary steps to wind down the initiative in accordance with the process outlined in the ROR operating organizations’ Memorandum of Agreement.

Sustainability

While ROR has always maintained transparency in terms of code and technical development, other aspects of its operations have not always been transparent and we therefore have taken deliberate steps to increase transparency on various fronts.

ROR’s curation process began transitioning to GitHub in 2020 and was made fully transparent and operational in 2022. Anyone with or without a GitHub account can access registry curation requests in the pipeline, view decisions that are made, and track when updates are implemented.

ROR’s product development roadmap is also publicly available on GitHub and users are invited to follow progress and submit issues for consideration.

The Memorandum of Agreement that outlines ROR’s operating and governance model is publicly available on the ROR website. The ROR website includes additional documentation about ROR operations, such as information about current team members and a timeline of how ROR first emerged. ROR has also begun posting annual financial reports on the website that go back to 2023 following the establishment of its current operating model in 2022. We do not currently make detailed financial information publicly available, such as staff salaries, or the amounts that individual organizations contribute to ROR.

Organizations and community members that participate actively in ROR operations are acknowledged on the website, including financial contributors, Curation Advisory Board members, and Steering Group members. We do not publicize the names of community members on our mailing list or those who attend our community meetings.

We continue to identify and consider future opportunities to make operations more transparent to ensure trust and also provide useful information for the community. Such areas could include documentation about how we run community meetings and structure community groups, incident response operations, and communication guides.

When ROR was first being established, it was resourced through a mixed and unpredictable funding model that combined in-kind support, grant funding, and community investments. As of 2022, ROR’s core expenses, including personnel, are currently funded through in-kind support from ROR’s operating organizations and are not dependent on any time-limited funds. ROR receives additional ad hoc contributions from community supporters but this is not our primary source of funding.

ROR does not charge for access and usage and does not intend to do so in the future.

ROR’s core operating costs have stable funding from its operating organizations, plus additional funds from organizations that provide ad hoc financial support for ROR. We maintain reserves to carry forward 1-2 years. We review our budget throughout the year to anticipate any volatility.

In addition to outlining support for ROR’s ongoing operations, ROR’s Memorandum of Agreement specifies procedures for winding down ROR operations and transferring assets as needed.

ROR’s financial planning and modeling processes are carried out by representatives of its operating organizations. ROR does annual and mid-year forecasting to anticipate any unforeseen expenses.

ROR aims to keep costs as low as possible while effectively and sustainably supporting core expenses and operations. ROR adheres to an informal policy of maintaining reserves to carry forward 1-2 years and we are working to codify this policy and make it more transparent and durable.

ROR has no revenue model and no current plans to offer a paid service; our objective is to continue offering ROR data and services freely available to everyone.

While ROR explored different potential revenue models during its startup phase, the ROR operating organizations made a decision in 2022 to fund ROR’s core operating expenses on a long-term basis so that ROR could continue to operate as a free and open service for everyone without being dependent on time-limited funding and without needing to develop a paid service model or distinct revenue stream.

ROR receives additional ad hoc funds from contributor organizations that believe in our mission. These are not directed contributions and there is no reward or “pay-to-play” option offered in exchange for these contributions. Such contributions help to offset ROR’s operating expenses and support time-limited activities. ROR does not depend on these contributions to be operational.

ROR has received grant funding in the past for time-limited activities. In the future, we may pursue additional grant funding for time-limited projects in accordance with our strategy and mission.

ROR makes both its data and services openly available for free. Data and code are made available under a CC0 waiver and MIT license, respectively. Our aim is to continue operating ROR as a free and open dataset and service.

While a value-added service model was considered after ROR’s initial launch, we have no current plans to develop any paid services. We may consider some service access constraints in the future such as API credentials to ensure overall stability and performance of our services, but such changes would not involve a fee model.

ROR has a dedicated team of paid staff members that provide support for ROR’s core operations. While ROR operates as a community-driven initiative, and regularly seeks community input and engagement to inform its operations and strategic directions, it does not rely on community volunteers for essential operations.

In the early days of the initiative, ROR relied on some community volunteers to advise on curation processes. We identified when this level of involvement was no longer viable and needed to be replaced with dedicated resourcing. We have not otherwise relied on volunteers for any core activities.

ROR’s Memorandum of Agreement supports continued staffing of core personnel roles so that we can ensure ongoing support for key functions. We have processes and logins documented and centralized to support continuity of institutional knowledge, and we have experienced staff transitions to test the quality of these safeguards, both in terms of staff members switching organizations and also leaving entirely. We maintain comprehensive onboarding and offboarding checklists for joining and departing team members, and documentation for new hires. Our technical processes are documented and regularly reviewed with backup support.

Like any small, mission-driven, community-oriented initiative, ROR depends on a limited number of passionate individuals who dedicate time and energy to their work and who have established trusted relationships with advisors, users, and stakeholders. While we have put several measures in place to make sure that we are not dependent on specific individuals such that we would be at risk of collapse, any departure of key personnel will always have an impact, and it would be naive to assume otherwise.

Insurance

All of ROR’s code and software processes are openly stored and documented on GitHub. Code is published under a fully permissible MIT License. We rely on additional tools and software to support our development work, such as hosting and logging. Wherever possible, we leverage open source components. We provide extensive documentation about how to access and query ROR data, and how to run the REST API locally, to enable other projects to use our data and code.

The ROR dataset is made openly available under the terms of a CC0 waiver and can be accessed via ROR’s data file or by querying the ROR API. ROR code is openly available and forkable under the terms of a MIT License.

No private or privileged information is included in the ROR dataset. ROR has a privacy policy published on our website that outlines how we handle private information, such as personal details of individuals on our community mailing list.

ROR data is openly available under a CC0 waiver and can be accessed in multiple ways, all of which are publicly documented. The ROR data file is published on the Zenodo digital repository and is subject to Zenodo’s preservation practices, which include regular backup, replication, and fixity processes, a retention policy of at least 20 years, and succession planning in the event of repository closure.

ROR code is available on GitHub under the terms of a MIT License and code repositories are automatically archived in Software Heritage. The archiving schedule is determined by Software Heritage. We may consider other automated or proactive mechanisms for code preservation moving forwards.

ROR is a fully open and public registry of factual information about research organizations. The metadata practices and curatorial stewardship that we apply to ROR registry data are not considered to be intellectual property of any ROR-related entity. Due to its factual nature, the information stored in the registry cannot be copyrighted or patented. In addition, ROR asserts no claims to registry data via the application of the CC0 1.0 public domain dedication.

ROR data can be accessed easily without proprietary extensions or software. For example, the ROR data file is available in both json and csv formats, and any integration can retrieve metadata via the REST API.

Europe PMC has long been at the forefront of ROR adoption. In their latest ROR-related project, they’ve used ROR in publication metadata to help their partner SciLifeLab, a major research facility and funder, make almost 17,000 linked research publications discoverable on Europe PMC. Read on to learn more about this important open metadata initiative.

Tell us about Europe PMC and SciLifeLab.

Europe PMC is hosted by EMBL’s European Bioinformatics Institute (EMBL-EBI), one of the six sites of the European Molecular Biology Laboratory (EMBL), an intergovernmental organization dedicated to molecular biology research, and Europe’s flagship laboratory for life sciences. SciLifeLab is a Swedish national center for large-scale research and one of the largest molecular biology research laboratories in Europe.

Building on an existing collaborations, the two organisations established a joint Memorandum of Understanding (MoU). As part of this the Europe PMC team has been working closely with SciLifeLab on opportunities to jointly strengthen support of open science.

How are you using ROR IDs?

Too often, research outputs aren’t linked to the organisations that fund them. Even when ROR IDs exist, they aren’t always connected to the relevant research, making it challenging to identify which organisations supported the work. This is the exact challenge SciLifeLab worked with Europe PMC to tackle.

SciLifeLab identified, through human curation, numerous relevant articles that they supported, which weren’t linked to them. SciLifeLab shared those article identifiers with Europe PMC, which linked their ROR ID to the relevant articles. This added ROR IDs to over over 16,500 items in the Europe PMC corpus, now openly available via the Europe PMC Articles API and through search on the website interface using the syntax ORG_ID:ror.org/04ev03g22.

How did you add ROR IDs?

The team at SciLifeLab have been developing ways to identify and link their research outputs such as software and data to publications. An initial step in this process is to identify their publication output, which is no small feat! SciLifeLab is not always included as an author affiliation so current advances in machine learning and AI would not identify this. SciLifeLab has an internal archive of publications that are produced by, or affiliated with, SciLifeLab, which is available via an published online web interface. 
In order to generate this list, human curation/reporting is required. This is a similar challenge shared by many institutions, the EMBL open science team included. Both EMBL and SciLifeLab will continue to send periodic updates to Europe PMC to keep the data current.

What else would you like to say?

Complete, open metadata is fundamental to how science is communicated, discovered and built upon. When the link between research outputs and the organisations behind them is missing, that chain breaks down. This work with SciLifeLab has strengthened the Europe PMC corpus with trusted, human-verified ROR ID metadata, and is a step towards a more complete and transparent scholarly record.

Joe Muller, Senior Publishing Technologies Developer at the Open Library of Humanities, tells us in this case study how and why the Janeway platform uses ROR, ensuring unambiguous author affiliations by retrieving ROR IDs from ORCID profiles as well as allowing authors to find their institution with a ROR-powered search.

As an open-access, scholar-led publisher, the Open Library of Humanities (OLH) should do everything we can to make sure metadata from our platform Janeway works seamlessly across the wide spectrum of places where published works might be discovered, read, and archived. The Research Organization Registry (ROR) provides the key to identifying the author affiliations attached to those works unambiguously, through the magic of uniform resource identifiers (URIs).

In the diamond open-access (OA) funding landscape, ROR also provides a key to understanding fair funding streams. Do the employers of Janeway authors contribute to Janeway’s development via membership in the OLH? Or are there imbalances, where institutions with a lot of Janeway authors and a lot of wealth do not contribute to a part of the diamond OA ecosystem they benefit from? We can crunch the data to find out, if we have unambiguous ways to link up a majority of articles in Janeway with their authors’ institutions.

How Janeway makes it easy for authors to use ROR

On Janeway, you interact with ROR when you record your institutional affiliation(s). Most often, that’s when you submit an article to be published on Janeway. You can search for the name or acronym of your institution to choose the correct ROR record. We display the institution’s name variants, location, website, and a link to the original record on the ROR website to help with tricky cases of identifying the right organization. You can put in multiple affiliations, and you can also enter a job title, department, and start and end dates, for most journals.

It’s even simpler if you have an ORCID with a public profile. Use the ORCID option to register on Janeway, and we will pull your latest public affiliation. If the affiliation has a ROR ID on ORCID, which most of them do, it will have a ROR ID on Janeway.

After publication, the ROR ID will be included in structured metadata streams such as Crossref DOI records, JATS XML files, and OAI-PMH feeds.

Scholars have to wade through so many digital systems to get their work done, funded, published, and recognized, that they have understandable fatigue with learning new systems.

If we make it easy for authors to include ROR-linked affiliations in their Janeway affiliations, they will. On the other hand, if we expect them to do careful data entry for a purpose that is only really clear in light of some abstract library science concepts (URIs and metadata interoperability), they may skip it, or potentially introduce errors. This is why we not only disallow manual entry of ROR IDs, but also bring ROR IDs over automatically from public ORCID profiles.

Of course, once the ROR-linked affiliation is in Janeway, we give authors ample time to see what it looks like in Janeway and make manual changes if needed. Most authors are very happy that it is there–they have already made the decision to make it public on their ORCID profile–and it is an extra bonus that they did not have to manually enter it again.

How Janeway integrated ROR

Our design process started as usual with user stories. Janeway editors talked to us about the features they wanted to see, and how they wanted them to work, as did librarians who manage Janeway installations at their institutions.

In this case, once we had a sense of user requirements, developer Steph Driver and technology director Andy Byers mocked up a few new possibilities for the author submission page. These went through several design reviews: our technology team, which included our UX researcher Katherine Parker-Hay and our publishing technologies librarian Siobhan Haime, thought through the implications for usability, accessibility, systems design, security, and maintainability.

I then got stuck in on actual implementation, beginning with porting our old data to the new data model, writing an interface for backwards compatibility, and creating a command-line interface for server administrators to load the ROR data dump into their Janeway installation and set it up to automatically update every month, with several engineering reviews by Andy and senior developer Mauro Sanchez. I then wrote the new user interface, incorporating a feature to implement the Contributor Roles Taxonomy (CRediT) started by Martin Paul Eve at the same time. I also wrote tools for auto-matching existing affiliations with ROR records. Finally, we made sure ROR data in Janeway sees the light of day, by updating our DOI deposits, metadata streams, reader interface, and plugins.

In the end, implementing ROR was absolutely worthwhile, and also a lot of work. This was not because of ROR – the ROR data and documentation are exemplary – but rather because of the need to balance an expanded Janeway data model with backwards compatibility for existing bulk import tools. We also needed to let any user enter and edit custom institution names as before, in case they do not yet exist in ROR, but keep them from changing the ROR-linked records in Janeway, since those names are provided by the ROR authority records automatically loaded in.

The rollout of the integration has gone well, beginning with OLH journals in mid-2025, and then going to our installations at universities in Darmstadt, Ghent, Huddersfield, London, Wales, Dublin, Massachusetts, and Virginia. Our development partner California Digital Library (CDL) also began their rollout by testing and contributing bug reports. The new user interface has been well received, with editors at Glossa and other journals chiming in with suggestions that we were able to implement quickly.

Since October 2025, 74% of articles published have ROR-linked affiliations. For records that we could auto-match in the OLH backlist, the number is 43%. Future plans include improving the auto-matching capabilities using ROR’s latest tools, adding support for translation of organization names in the user interface, and adding features for coauthors.

In the ROR Annual Community Meeting in February, we celebrated seven years of ROR and the ROR community that makes it all possible in three terrific online sessions featuring ROR team members and ROR enthusiasts from all over the world. If you missed any of the sessions, read the highlights and access slides and recordings below, and be sure set yourself a reminder to attend live next year in early 2027!

Reflections on where ROR is now

The annual ROR Community Update is always an opportunity for reflection on how far we’ve come, but in this seventh edition of the meeting we also took the opportunity to reflect on what has remained the same. In the last seven years, ROR has changed a lot in terms of how we’re used, how we operate, and what we offer, but we’ve remained true to our guiding vision and values. And of course, one thing that will never change is our perpetual gratitude to all of you around the world who support ROR in so many different ways: from submitting registry updates to integrating ROR in your systems and from telling colleagues and friends about ROR to investing in our sustainability.

Highlights from the community update:

• Key ROR system integrations in 2025 include Symplectic, RAID, DOCID, RDA, and OJS 3.5.
• Crossref also added support for ROR IDs as funder identifiers, and entities such as Fonds de recherche du Québec and the Worldwide Protein Data Bank registered thousands of new Crossref records with ROR IDs.
• The number of Crossref records with ROR IDs more than tripled in 2025, and the rate of ROR use in DataCite and ORCID records also remains very high.
• In 2025, ROR’s curators processed over 12,200 user-submitted curation requests, adding over 8,500 new relationships and expanding domain metadata.
• ROR curators focused on streamlining workflows and increasing global coverage in 2025; notable regional growth in ROR includes East Asian research organizations.
• ROR’s technical team deprecated v1 of the ROR schema and API in 2025, improved the performance and stability of the ROR API, and launched a new “single search” name-to-ROR-ID matching strategy.
• Technical goals for 2026 include improving curation workflows, enhancing the web search experience, upgrading core technologies, and making single search the default affiliation matching strategy.
• The timeline for requiring a client ID in order to receive the current ROR API rate limits has been extended to Q3 of 2026.

ROR, funders, and open funding metadata

One of those aforementioned changes in the last seven years is that ROR is now often used as an identifier for organizations that fund research as well as for organizations that perform research. The annual meeting session How ROR Can Help Research Funders offered several perspectives on ROR’s usefulness in helping funders track the research that has resulted from their support.

Highlights from the funder session:

• Hans de Jonge, Head of Open Science Policies, NWO, made a compelling case for the general need for open funding metadata in scholarly publishing. Shortly after the annual meeting, Hans also wrote a blog post for us explaining how and why NWO has integrated ROR into its open API.

• Ana Van Gulick, Head of Customer Engagement for Government, Funders & Nonprofits, Digital Science and (in absentia) Ishwar Chandramouliswaran, Lead Program Director, Office of Data Science Strategy, NIH, both of the GREI initiative, explained how and why the seven participating generalist repositories implemented ROR and announced some exciting forthcoming ROR-powered metrics and metadata dashboards.

• Ami Gala, Sales Director, Americas, Altum, gave a detailed overview of how and why the popular Proposal Central grants management system uses ROR.

ROR and National Research Strategies in the Asia-Pacific Region

As we mentioned in the Community Update, ROR worked hard in 2025 to make ROR a truly global registry: of the 8,000+ organization records we added to ROR last year, nearly 20% represent organizations located in East Asia. The annual meeting session The Role of ROR in National Research Strategies therefore featured speakers from Japan and Korea as well as from Australia and was held at an APAC-friendly time.

Highlights from the session on national research strategies:

• Kazuhiro Hayashi, Principal Senior Fellow, NISTEP, gave an overview of local PIDs in Japan, explained recent collaborative work to link the NISTEP Dictionary and ROR, and pointed to ROR as an example of an important PID in the proposal for the next STI Basic Plan of the Science Council of Japan.

• Clare Nulley-Valdes, Assistant Director Evaluation and Impact Policy, Australian Research Council, spoke on Australia’s National PID Strategy and ARC’s PID Action Plan and reported that ARC has incorporated ROR IDs into its internal systems and will begin requiring researchers to use ROR in funding acknowledgements.

• Jungwoo Lee, Senior Researcher, KISTI, discussed the challenges of building and maintaining an in-house organization registry, reported on a recent ROR / KISTI collaboration to improve coverage of Korean organizations in ROR, and explained how KISTI uses ROR and Web of Science to track Korean research.

Looking ahead

Change is one of the few constants in life, but what won’t change is that ROR will always be community-led and will always be committed to improving our content and services based on your feedback. We’ve got plenty to do in the months and years ahead, as you can see from our roadmap, but we’re still always happy to hear from you about what else you’d like to see us do and how we can better support your needs for open, high-quality, interoperable organization information. Join us in 2026 at our bi-monthly Community Calls and other events and write us at support@ror.org with questions and suggestions.

The Dutch Research Council (NWO) announced recently that it has integrated ROR in its open API. Hans de Jonge, director of Open Science, tells us why and how NWO uses ROR in this latest installment of our case study series.

See also Hans de Jonge’s recent presentation at the ROR Annual Meeting session How ROR Can Help Research Funders.

What made you decide to use ROR?

NWO is the major research funding council in the Netherlands with a longstanding commitment to Open Science. About five years ago, we decided we needed to apply the principles of Open Science not only to our grantees but also—as much as possible—to our own data. We decided to move away from using proprietary data for our annual open access monitoring and use open metadata only.

We developed an open API which makes structured data for all our funded projects and their outputs openly available as open data for anyone to use. Universities use this data for instance in developing institutional dashboards about the external funding they acquire, but the data is also available for researchers.

We also developed a PID strategy for our organization, outlining that over time we would implement DOIs for outputs, ORCIDs for researchers, Crossref Grant IDs for our awards, and ROR for organizations. We have now integrated ROR into our API, meaning that the affiliations of all researchers in that database (PIs as well as their collaborators) are now identified with ROR.

How do you use ROR?

For the moment we only use ROR in our NWOpen API. In the back end a database is running which converts organizational information to ROR IDs. The challenge here was that our own organisation master list included both top-level organisation names and child organisations. Quite a bit of manual validation came into this.

The ambition is to integrate ROR more fully in our grant management system. Like many national funding councils, we have a custom-built grant management system called ISAAC, and we are currently in the process of renewing that system. PID integrations (ROR, ORCID and Grant ID) are high on the list of requirements.

For now we hope that exposing ROR through our NWOpen API will already help users to better identify organizations. Our old field consisted of a mix of top level organizational names and faculties and departments (“University of Utrecht | Faculty of Veterinary sciences | department Clinical Sciences”). Introducing ROR allows people to just search for Utrecht University’s ROR and retrieve all funded projects.

We are also considering exposing ROR on our project website.

What were the steps you took to integrate ROR into your systems and workflows?

The honest answer is that it took us quite some time. The difficulty was that it collided with other data-related projects within our organization. But we are very proud to have pulled it off, because it immediately shows people within and outside of our organization the enormous potential of persistent identifiers. We experienced the same thing when we introduced Grant IDs two years ago.

Anything more to say about ROR?

We are proud to join the growing ROR community. Persistent identifiers are a key enabler in the world of scholarly communication and open science. We are very well aware that we as a funding council have a role to play – not only in using PIDs to our own benefit but contributing to this landscape by integrating them into our systems and exposing them so all these entities can be better connected to the benefit of an open scholarly ecosystem.

This time last year, when we reflected on ROR’s milestones in 2024, one of the major highlights was the launch of version 2 of the ROR schema and API. Now, as we think back on 2025, where we were a year ago makes a nice bookend to one of the major highlights of the current year: last week’s sunset of version 1 of the ROR schema and API. Developing and launching new products and services is both strategically important and satisfying, and we’ve certainly done that in 2025, but as any initiative matures, it also becomes essential to put away certain things, clearing the path to sustainable maintenance and renewed innovation.

ROR, in short, is fully grown now: we’re well-established as the obvious and trustworthy choice for research organization identification, we’re processing more and more (and more and more) requests for changes and additions to the registry, and we’ve established systems and processes that let us run like a well-oiled engine.

Fulfilling its original purpose, ROR plays a key part in providing open affiliation metadata for numerous workflows and use cases: to ensure research integrity, manage transformative agreements, track research output, and analyze institutional collaborations. ROR IDs have also proven to be valuable for many more use cases and types of users than anyone originally envisioned. Notably, both publishers and funders have begun using ROR IDs to identify funders, and we expect this use of ROR to increase dramatically in the future, especially as more and more funders register DOIs for awards.

Here’s a summary of what ROR did in 2025.

Refashioning ROR’s schema and API

Much of the technical work we’ve done this year was dedicated to infrastructure and service-level improvements intended to keep our API fast and reliable and our metadata schema comprehensive and flexible in the future.

At the end of July, we changed the default response to ROR API queries to return results in version 2 of the ROR schema. This significant change went very smoothly, with the vast majority of ROR API users having prepared their ROR integrations for the switch before this date. With this first phase of version 1 deprecation in place, we continued to remind ROR users that version 1 would be sunset in December. By the time the sunset date of December 9th rolled around, fewer than 8% of ROR API users were using version 1. As of this writing we still haven’t received any reports of broken userspaces, which makes us very happy! The current version of the ROR API and schema was designed in collaboration with our community for long-term support. We are glad that its more descriptive and useful form will be the default experience for all users.

Another major technical initiative this year has been the implementation of a new strategy for affiliation string matching, which we announced in early December. This new strategy, developed in collaboration with Crossref’s Director of Technology, Dominika Tkaczyk, is both faster and more efficient than our existing matching, while also achieving better performance for this task. As such, it is particularly well-suited for large-scale affiliation string matching and was adopted by OpenAlex at the start of the year to significantly improve their ROR ID coverage. We look forward to hearing more from ROR API users about this new matching strategy in 2026, which we plan to make the default next year.

Ensuring high-quality open metadata through community curation

We continue to see logarithmic growth in requests to create new ROR records and modify existing ROR records, and while we still process many requests from individuals and individual research organizations, we are working more and more with national and international research information initiatives on large-scale curation projects. This year, for instance, we collaborated closely with the Korea Institute of Science & Technology Information (KISTI) on a project to add and modify approximately 450 Korean research organizations, continued our work with the Fundação para a Ciência e Tecnologia to add and modify hundreds of records for research organizations in Portugal, and added and modified many ROR records for African research organizations in collaboration with the Africa PID Alliance.

Just a few of ROR’s other collaborative curation projects in 2025 include the following:

• Adding new Category 2 Centres for UNESCO
• Working with Japan’s National Institute of Science and Technology Policy (NISTEP) to reconcile ROR records with the NISTEP Dictionary of Names of Universities and Public Organizations
• Adding and updating research infrastructure facilities for Australia’s National Collaborative Research Infrastructure Strategy (NCRIS)
• Working with The National Library of Luxembourg to add and update Luxembourgian organizations
• Adding Czech Large Research Infrastructures listed in Czechia’s national “Road Map of Infrastructures”

If your organization would like to work with ROR on a national-level project, you’re always welcome to use our bulk processing spreadsheets to send us a large number of requests for additions or updates.

Supporting new users and uses of ROR

In 2025, we saw a marked increase in the use of ROR to address key use cases for funding organizations and for funding metadata. Open Funder Registry records have been incorporated into ROR, and ROR is recommended in both the Crossref metadata schema and the DataCite metadata schema as the preferred identifier for funding organizations. Community initiatives such as the Barcelona Declaration on Open Research Information are also emphasizing the importance of ROR in bringing broader visibility to open funding metadata to enable downstream reporting and tracking of research.

Soon after Crossref announced that its members could now use ROR IDs to identify funders in any type of content, for instance, Québec’s premier funder Fonds de Recherche du Québec (FRQ) became an enthusiastic adopter of ROR. We worked with FRQ to add and modify ROR records representing the affiliated institutions of the principal investigators on FRQ grants so that FRQ could register more than 20,000 grants with Crossref, using its own ROR ID to identify itself.

Throughout 2025, we’ve also continued to improve the quality of metadata in Chinese-language records identifying funders. Recently, in advance of the Open Science Framework’s transition to using ROR IDs for funders, we’ve also been working with OSF to review nearly 150 funders and match them to ROR IDs or add them to ROR. We anticipate even more of this work next year as more systems continue to use ROR IDs to identify funders for all kinds of DOI content types, including grants.

As for publishing systems, we were thrilled to see the release of PKP’s Open Journal Systems (OJS) 3.5 in which ROR IDs for author affiliations are now part of the core codebase instead of being managed by a plugin. OJS is used by thousands of small publishers all across the globe, and this update means that these small publishers can more easily identify author affiliations in DOI metadata. Another notable publishing integration is in the Janeway system created by the Open Library for the Humanities, where authors who use an ORCID ID to log in during the manuscript submission workflow will see their institutional affiliation automatically added thanks to Janeway’s ability to pull that information from the ORCID API using the ROR ID. Both the ORCID ID and the ROR ID are then included in DOI metadata.

Publishing systems certainly aren’t the only ones to use ROR. CRIS systems are increasingly standardizing on ROR, including Symplectic Elements, which began to enrich and normalize organization data with ROR this year. We’ve also seen the Research Activity Identifier (RAID) initiative to identify and register research projects using ROR for organization identifiers in its schema and the Research Data Alliance incorporating ROR into its member registration form in a thoughtfully-designed user interface.

Contributing to key conversations and initiatives

As a community-driven initiative ourselves, we recognize how important it is to do professional service for others and participate in collaborative activities and discussions. In 2025, ROR’s participation in such work extended well beyond our own Community Calls and events. ROR team members serve on working groups and advisory bodies for the Barcelona Declaration, the National Information Standards Organization (NISO), OpenAlex, the Public Knowledge Project (PKP), and Collaborative Metadata (COMET), just to name a few. As a selected infrastructure in the Global Sustainability Coalition for Open Science Services (SCOSS), we are also involved with SCOSS events and activities, helping to support the global open infrastructure that makes so many research information services possible.

We also participated this year in the final NSF-funded workshop on FAIR Facilities and Instruments, offering our perspective in particular on how research facilities can use persistent identifiers to help their important work gain better recognition. Finally, we presented at global conferences such as euroCRIS, Open Repositories, the Society for Scholarly Publishing (SSP), and the Japan Open Science Summit, reaching a range of communities through these events.

As 2025 comes to a close, we’d like to thank all those who have helped bring ROR to this point. We remain grateful for your questions, your requests, your comments, and your contributions, and we remain committed to learning from and supporting the ROR community in 2026, starting with the ROR Annual Community Meeting sessions on February 3rd and 4th, 2026. See you next year!

In the past few years, a great deal of attention has been paid to the importance of using ROR IDs to identify organizations that researchers are affiliated with. Less well-known, but equally important, is that organizations can also act as creators or contributors themselves, and these organizational creators and contributors can also be identified with ROR IDs, which is particularly useful for tracking organizational partnerships.

Organizational Partners

DataCite now has over 1,700 members and, as the membership expands, different types of organizations join the DataCite community. Some of these new members are organizations with important partnerships with other organizations. The DataCite metadata schema for DOIs includes the capability to define a type for creators and contributors which can be Personal (default) or Organizational. Personal creators and contributors are typically identified with ORCID iDs, of course, whereas Organizational creators and contributors are identified with ROR IDs.

An organization might be listed as a creator of or contributor to a dataset for a number of reasons, many of which reflect organizational partnerships. An organization that hosts datasets for another organization, for instance, might be listed as an organizational contributor in the metadata for that dataset. How are current DataCite members using ROR IDs to identify and track partnerships with other organizations?

The EarthScope Consortium operates major scientific facilities focused on Geophysical Sciences, primarily Geodesy and Seismology. These organizations existed independently as UNAVCO, which operated the NSF GAGE facility, and Incorporated Research Institutes for Seismology (IRIS), which operated the NSF SAGE facility. As of October 2025, these entities merged into the National Geophysical Facility with funding from the U.S. National Science Foundation. Since 2016, both facilities have used DataCite DOIs to identify datasets, and they manage significant repositories. The legacy names of these repositories in DataCite metadata are iris.iris (for the NSF SAGE facility operated by IRIS) and unavco.unavco (for the NSF GAGE facility operated by UNAVCO).

EarthScope worked with Metadata Game Changers to increase utilization of identifiers in both repositories during 2025 (Habermann and Riley, 2025). Over 450,000 identifiers were added to several fields in the EarthScope repository during this project. It turned out that 97% of the creators in the NSF SAGE repository were organizations, so it provided an opportunity to explore how ROR IDs can be used to track organizational partnerships.

Table 1 shows the organizations that are included as authors in the IRIS repository and the number of times they occur. Together, these organizations account for over 95% of the datasets in this repository. “IRIS DMC” and “Incorporated Research Institutions for Seismology” are two legacy names for the same host facility used in the metadata, as indicated by their association with the same ROR ID. The other organizational creators include universities and government agencies that produce specialized geophysical datasets.

For example, the Global CMT Project calculates Centroid Moment Tensors that give key source parameters like magnitudes, fault orientations, and locations for earthquakes all over the world while Global ShakeMovies generated by Princeton University and given DataCite DOIs are used to visualize ground motion as seismic waves travel around the world.

All but one of the organizations listed in Table 1 had ROR IDs when we started this project, and these were added to the DataCite DOI metadata as creator identifiers. The Global CMT Project became a community facility during 2020 and a ROR ID was created for the facility during this project, making it possible to add a creator identifier to over 72,000 datasets created by The CMT Project.

DataCite Commons is a great tool for browsing resources associated with organizations using their ROR IDs. The DataCite Commons Earthscope page shows over 124,000 works for this organization (Figure 1, left).

The creators and contributors list shows only creators with ORCID iDs, i.e., people, but the works associated with the organizations in Table 1 can be found using the names of the organizations in the Filter Works box (green box). For example, filtering works for “Princeton” yields 9,020 works, the shake movies in the EarthScope repository authored by Princeton. Clicking the title for a work shows the creator tab which includes Princeton as a creator of the dataset.

The fact that ROR IDs of creators appear in the Commons output for the repositories that hold the resources and mint the DOIs is great, but it is expected. An unexpected benefit of these ROR IDs is that they create a two-way link between the repository and the partner organization. This link is visible through the DataCite Commons page for Princeton which shows over 13,000 works (Figure 1, right). Filtering these for “ShakeMovie” returns the Global ShakeMovies in the EarthScope repository. The link added to the EarthScope metadata makes the connection to Princeton that is discovered by DataCite Commons.

At the time that these data were collected, the Princeton ROR ID was associated with about 13,000 items discovered by DataCite Commons. Over 9,000 of these, i.e., 68%, are the shake movies in the EarthScope repository. The amazing thing is that these items showed up on the Princeton DataCite Commons “profile” automagically – without Princeton doing anything. This behavior is like items added to DataCite showing up on a researcher’s ORCID profile, but for organizations instead of individuals. The DataCite Commons page for an organization ROR acts like an ORCID profile for an individual.

Figure 2 shows partner resources in the EarthScope repository and in the partner Commons pages. EarthScope resources represent over 68% of DataCite resources for many partners. In the scenario described here, EarthScope has provided data management services to partners for many years. Adding ROR IDs to the EarthScope metadata for these datasets adds an increasingly important connection service to these partnerships.

We can explore organizational creators in other repositories using this DataCite API query:

https://api.datacite.org/dois?&affiliation=true&publisher=true&page%5Bsize%5D=1&query=creators.nameType:Organizational

Results from this query indicate that almost 1.4 million records in DataCite have organizational creators. Table 2 shows the ten repositories with the most organizational creators and the number of those creators in each. EarthScope just makes it into the list at number ten!

In a sample of 10,000 records with organizational creators from each of these repositories, the Distributed System of Scientific Collections (DiSSCo) is the only one besides the SAGE repository that includes identifiers for their organizational creators. The most common creator ROR ID in DiSSCO (https://ror.org/0566bfb96) identifies the Naturalis Biodiversity Center, a biodiversity museum in The Netherlands that coordinates closely with DiSSCo. The DataCite Commons page for The Naturalis Biodiversity Center currently shows 7,036,619 works associated with this ROR ID, with 99% of them created in a major expansion of the DiSSCo repository during 2025. This is a clear bright spot that uses ROR IDs effectively to recognize their partners!

Conclusion

ROR IDs can serve as identifiers for research organizations in several roles. Historically attention has focused on organizations that researchers are affiliated with, but organizations can also act as creators or contributors themselves. This information can be reflected in research metadata such as in the DataCite metadata schema for DOIs. Over 1.4 million DataCite resources have organizations as creators and over 44 million have organizations as contributors, providing rich opportunities for DataCite members to take advantage of ROR IDs in their metadata.

Eight organizations account for over 110,000 resources in the EarthScope repository and seven of these had ROR IDs at the beginning of this project. We worked with ROR to create a new ROR for the remaining organization (the Global CMT Project) to complete the identifier coverage.

Adding identifiers for these organizations to the EarthScope repository represents an important “connection service" provided to these partners by EarthScope and inherent in the two-way nature of ROR IDs and other identifiers. This kind of connection occurs for all ROR IDs included in DataCite DOI metadata, and DataCite Commons provides an auto-update of resources associated with organizations like the auto-update capabilities for ORCID iDs for researchers. If you are an organization that hosts, creates or contributes to datasets that have DOIs registered in DataCite, make sure your ROR ID gets into the metadata so that your resources are visible through DataCite Commons and discoverable in DataCite’s APIs.

We are delighted to announce the launch of a new matching strategy in the ROR API to help you match complex affiliation text strings to ROR IDs at scale.

Publishing and repository systems have traditionally stored author and affiliation information as unstructured data, capturing intermingled details about an organization as a single text string in a single field. This unstructured information makes it difficult to reliably connect research outputs with associated institutions.

Many repositories and publishers are beginning to use ROR instead, recognizing the importance of producing and sharing open, structured affiliation metadata, and here at ROR we continue to encourage Crossref and DataCite members in particular to include ROR IDs in DOI metadata. Often, a crucial step for Crossref and DataCite members in ROR adoption is to convert text strings extracted from submitted manuscripts or stored in legacy data to ROR IDs, and this is one reason the affiliation matching endpoint is the most-used endpoint of the ROR API.

We know that accurately matching text strings to ROR IDs is a priority for many users, and therefore we have been continuing to develop ways of meeting this need with our API and with other services.

How and why we developed the new affiliation matching strategy

ROR’s first affiliation matching strategy, developed by Crossref’s Director of Technology, Dominika Tkaczyk, was announced in 2019. It has been available in the ROR API ever since. This strategy works by breaking long affiliation strings into substrings and making multiple queries against the ROR registry. If a match is found, the ROR API returns the ROR record identified with a chosen:true indicator.

Beginning in 2023, the ROR team began collaborating with Crossref again to develop a new matching strategy that would be both faster and more precise at scale. Throughout 2024, the principles behind this collaborative research were explained in a popular series of blog posts about matching on the ROR blog co-written by ROR Product Manager Adam Buttrick and Dominika Tkaczyk.

Early in 2025, we reported that this collaboration between ROR and Crossref had produced a new, faster, more precise strategy for matching affiliation strings to ROR IDs. Unlike the original “multisearch” matching strategy, the new strategy performs a single search of the ROR registry to find the most likely match, and therefore we’ve dubbed it “single search.”

OpenAlex quickly incorporated this new strategy into its own affiliation matching, and Crossref used it to produce a dataset of 95 million affiliation strings from its metadata matched to ROR IDs. At ROR, we then began the work to include this strategy in our own API, which was recently completed and released last month. Like all of ROR’s code and data, the single search strategy is freely and openly available.

Benefits of the new affiliation matching strategy

In a demonstration of the single search strategy given at the November 2025 ROR Community Call, Adam Buttrick explains that the new strategy has been designed to reduce the rate of false positive matches, because we know that accuracy is of primary importance for unsupervised automated matching. The single search strategy is more computationally efficient, better able to find relevant results, and more accurate than the already high-performing multisearch strategy.

In just one of several examples drawn from real Crossref metadata, Adam shows in the below video that a string such as “Department of Cognitive Sciences, School of Social Sciences, University of California, Irvine, California;” is not matched by the current multisearch affiliation strategy on the ROR API but is correctly matched by the new single search strategy to the ROR ID for the University of California, Irvine: https://ror.org/04gyf1771.

Using the new affiliation matching strategy

To try out the new single search matching strategy, append the term &single_search to a ROR API ?affiliation query as described in our updated documentation.

Note that the single search strategy is currently in a preview period, so the existing multisearch matching strategy is still the default for ?affiliation queries. Both matching strategies will remain available in the ROR API for the foreseeable future, but we plan to make the single search strategy the default for ?affiliation queries in the first quarter of 2026.

To give us feedback, you can open an issue on our roadmap or contact us at support@ror.org. Please note that any individual string might be less well matched using the new strategy than using the existing strategy, so we are most interested in tests that use sizable open datasets.

ROR’s essential purpose is to improve metadata throughout the scholarly information ecosystem so that reliable connections can be made between organizations and the research they support. Clean, normalized, structured, open organization information helps ensure research integrity and enables better tracking and analysis of research. Our hope is that by constantly improving ROR’s systems and services we can improve scholarly metadata for everyone.

What made you decide to use ROR?

Cambridge University Press originally adopted GRID. After ROR became the primary community-driven, open registry of persistent identifiers (PIDs) for organisations producing research, Cambridge adopted ROR IDs into publishing workflows that require a unique identifier for institutions. A PID for institutions associated with the research we publish makes publishing services more efficient, streamlines business analysis and is essential for accurate business insights.

As the academic publishing industry pivots towards an Open Access publishing model, PIDs are becoming even more crucial in the nuanced delineation of research funding bodies, institutions funding the publishing of academic research and the relationships they have with the authors conducting the research.

How do you use ROR?

ROR is a critical facet in our analytical toolkit; understanding the totality of our relationship with a research body regardless of how we interact with them is impossible without a persistent identifier drawing our immense and very different data holdings together. As developments in AI continue to influence present strategy and future thinking on the efficacy of publishing models, bad data becomes a business risk. Too often, data attributes associated with research bodies are uncertain, inadequately classified and not easily resolvable. Poorly maintained data holdings prevent the dynamic opportunities AI offers to the industry.

Using ROR as a PID in our systems redefines our data; it becomes a new, exciting asset rather than an artefact of the publishing process. PIDs are a simple, yet very powerful catalyst to creating insights which enables publishers like Cambridge to make objective data-driven decisions; to define, assess, refine and deliver strategic goals and to make publishing programs as successful, rigorous, efficient and cost-effective as possible.

What were the steps you took to integrate ROR into your systems and workflows?

The use of ROR in our publishing processes and the application of it to our data holdings is an ongoing process. It affects multiple workflows, databases, repositories, dashboards, publishing tools, and extensive examination of our data holdings associated with research institutions across systems used for different aspects of publishing and providing access to research.

Cambridge, like many academic publishers, has existed for quite some time and our data holdings are vast, so our data strategy adapts all the time as we assess each system, repository and data asset. Cambridge is modernising and upgrading many of its publishing tools; making sure the metadata that defines our relationships with publishing partners is unambiguous is a priority.

Over the years, institutions have been referred to in myriad ways, for lots of reasons, not least system variances, interactivity with other technologies, human intervention and the evolving cultural, historical and political environments academic research organisations exist within. The relationships are not always obvious, but this is precisely what the use of a PID like ROR is attempting to overcome.

What else would you like to say about ROR?

Cambridge is proud to be a member of the ROR community and to contribute to the adoption of ROR by associating it with the metadata for our published research and incorporating it into our workflows. A global, open, community-led registry that enables precise associations between researchers, their research and the institutions associated with it naturally allies with our continuing commitment to being a diligent contributor to the academic publishing community and helps us to realise our goals and ambitions as an Open Access publisher.

Have you ever wondered exactly what happens once you request a new ROR record or suggest a change to an existing ROR record? In this blog post, we take you through all the steps involved in ROR’s open, community-driven process for making sure that the information in the Research Organization Registry is complete and accurate.

ROR’s community-driven process for maintaining high-quality information about research organizations in our registry means that anyone can request a new ROR record or suggest a change to an existing one. Our curatorial process for assessing these requests involves both human and automatic review, and all this review happens in the open so that anyone can follow along. Here’s a deeper dive into how that works.

Step 1: You submit a request

Requests come in two main forms: individual requests and bulk requests. In addition to processing requests, ROR curators also carry out independent quality assurance projects, which fall outside the scope of this post.

Individual requests are typically submitted through our public curation request form. The two most common actions are to add a new organization or change an existing record; however, you can also request to deprecate an existing ROR record, merge two or more ROR records, or split an existing ROR record into two or more records. Whatever the request type, the best requests have the most complete information possible, which speeds up processing time. There is a human behind every review, and so the more information is sent, the more context ROR curators have to process the request, and the less time they have to spend seeking further information.

For new record requests, it is particularly important to provide multiple DOIs or URLs for research publications that use the organization as an affiliation or that acknowledge the organization in some way, such as for funding, in order to meet ROR’s definition of what constitutes a “research organization.” For all other types of requests, providing a link for evidence that a change should be made, such as a name change announcement on the organization’s website, is also very helpful for faster processing.

If you are having difficulty submitting through our form, you can always create your own request directly on GitHub if you have an account. We have templates available for adding a new organization or modifying an existing record.

Bulk requests for larger sets of organizations can be submitted via email through a spreadsheet template. There are two spreadsheets available—one for adding new organizations and one for updating existing records—so please choose the correct spreadsheet for your requests or send us both spreadsheets if you have both new and update requests. Instructions for how to complete the spreadsheets are in the headers of each column.

Step 2: A GitHub issue is created for the request

Behind the scenes, your request is automatically converted into a GitHub issue. This means it’s immediately visible in our public curation updates tracker, where anyone can see requests and anyone with a GitHub account can comment. Because transparency is central to the ROR model, all requests and discussions are open by default. Personal information from the requester never gets published or shared. ROR currently receives approximately 2000 individual submissions per month in addition to bulk requests of various sizes. Processing for individual requests takes about four to six weeks, depending on complexity.

For bulk requests, a single project issue is created, and when it is processed, each row in the spreadsheet is converted into individual sub-issues linked to the project. Because bulk requests range in size from tens to thousands of requests, they can take a longer time to process compared to individual requests.

You will receive an email confirmation with a link to your new issue on GitHub where you can follow along for next steps. If you have any additional context later, such as additional links to research or a correction to the form, please comment on your GitHub issue. Again, this requires a GitHub account, but they are free and easy to make. As always, you can also email support@ror.org if you have questions or additional information at any point in the process.

Step 3: Automatic tools do initial triage of the request

Next, ROR’s automated tools perform initial checks on the request. These are scripts and workflows we’ve developed that take the form of “bots” that you might see interacting with your GitHub issue. A ROR curation bot helps us format the human-submitted data in the request by deleting extra fields, fixing formatting so it’s ready to process, and adding language tags. You can see the changes this bot makes in a comment below the issue, where the red highlight with the minus (-) symbol indicates what’s been deleted and the green highlight with the plus (+) symbol indicates what’s been added.

ROR curation bots also find information that helps with curator review and metadata quality. For example, you might see a bot find a GeoNames ID for the city and country provided, flag potential duplicates in ROR, check for additional identifiers like an ISNI or Wikidata ID, or look for research where the organization is used as an affiliation.

These automated steps help ROR curators spend their time on meaningful review rather than on data clean-up. Our curators are always looking for ways to improve these processes by refining and building new curation bots and scripts, all of which can be found in our Curation Ops GitHub repository.

Step 4: Curators review the request

After automated triage, human curators step in to review the request in detail to ensure it is appropriate for ROR’s scope and consistent with other records. Metadata in each issue is reviewed and corrected to conform with ROR’s metadata policies and issue formatting.

ROR’s definition of “research organization” is broad by design: ROR strives to be a truly global registry, one that includes research organizations of many kinds from all over the world, so that these organizations can reliably be connected to the research outputs and activities they support. At the same time, ROR also upholds a high degree of consistency and integrity of the registry as a trusted and widely adopted source of institutional data. Therefore, for new organizations to be included in ROR, one of the most important criteria is whether the organization is cited in published research.

An organization should have clear reference to its activities in sources like contributor affiliations and funding acknowledgements in public research outputs to be considered a “research organization” that is in scope for ROR. If no links to research are provided in the request for a new ROR ID, if only links to self-published research are provided, if the organization is neither used as an affiliation nor acknowledged as a source of funding in open research repositories and knowledge graphs, and if no research is listed on the organization’s website, the request to add that organization to ROR will be declined. Of course, organizations may also reapply once ROR requirements are met.

In addition, since ROR IDs are persistent identifiers intended for long-term stability, ROR IDs identify organizations with long-term stability. Thus, ROR also requires that an organization is used as an affiliation in published research outputs by multiple people, indicating that it is not a one-person organization or consultancy that will not exist past the lifetime of a single person. Similarly, ROR expects instances of mention in multiple research outputs. Requests that have been declined can always be reconsidered if the requestor can provide evidence of multiple instances of research usage with multiple authors.

For changes to existing organizations, we verify the information against reliable public sources like the organization’s website, authority files, and national-level registries; we sometimes also reach out to the requester for clarification. The updates are then coded in the issue itself. You will see a string following “Update:” for changes to the record or “Related organizations:” for changes to the record relationships.

Any approved request will be moved to “Ready for sign-off / metadata QA” in our ROR Updates project board in GitHub. When a request is ambiguous or complex, it may be marked as “Second Review” or “Needs Discussion” and referred to the Curation Advisory Board for asynchronous review. The board also meets monthly to discuss tricky cases and broader policy questions.

Read more about our scope, criteria for inclusion, and curator workflows.

Step 5: Curators prepare the release

Every two to four weeks, all reviewed and approved requests are grouped into the next ROR data release. All metadata is double-checked using both automated scripts and manual review. This ensures that names, identifiers, and other fields are consistent across the registry and that everything aligns with ROR’s metadata policies. Additional checks are run to ensure there are no duplicates and that the metadata passes various quality and integrity validations.

Metadata from issues is then extracted programmatically to create new and updated records, turning the approved changes into JSON files that follow the ROR format. These files are then further refined in our release data pipeline, where we do things like update the date last modified and build new relationships between organizations that are again reviewed. Once the draft release passes those checks, we upload it to the staging environment to test it to make sure the new and updated records show up correctly in the API and search before moving everything into production.

Step 6: A new release of ROR is published

And then, hooray! A new release of ROR is published and becomes part of the open, global registry.

The release is then announced across our community channels, including on social media – LinkedIn, Bluesky, and Mastodon – and the ROR Technical Forum. After the release goes live, you’ll see a comment on your GitHub issue letting you know it’s been published.

The best requests

Don’t forget that the best way to ensure faster processing of your request is to submit accurate, clean, and complete metadata! Please follow along on the journey of your own requests directly on GitHub to see updates, comments, and the decision-making process described here. We are always available to help with questions about requests at support@ror.org. In our next post in this series, we’ll go into more detail about how to submit an accurate, clean, and complete ROR request!

Your requests help make ROR great

Now, the next time you submit a new request or think about suggesting a change, you’ll know exactly what happens behind the scenes. By keeping curation open and collaborative while applying careful checks and rigorous processes, ROR ensures that the registry remains complete, accurate, and trusted by the entire research community.

We are thrilled to introduce the newest member of the ROR pride: Joseph Rhoads joined the ROR team in September as our new Technical Lead.

Joseph follows in the footsteps (pawprints?) of long-time Technical Lead Liz Krznarich, who supported ROR’s foundational technical development over the past several years to strengthen and scale the registry’s infrastructure and implement key initiatives such as our independent curation workflow and version 2 of the ROR schema and API. Earlier this year, Liz embarked on a new endeavor in the world of geology and natural history to be a software developer for the Wisconsin Geological and Natural History Survey (ROR ID https://ror.org/02kj3rm24) in her home state. We are immensely grateful for Liz’s many contributions to ROR over the years, and for setting ROR up for sustained technical stability and success into the future.

Originally from the US but currently based in Valencia, Spain, Joseph joins ROR from DataCite, where since 2021 he has been supporting software engineering various DataCite technical projects and services, including DataCite Commons. Prior to joining DataCite, Joseph worked at Brown University Library, where he gained extensive experience in repository management and institutional research monitoring, both of which will be valuable perspectives to bring to ROR. Joseph will continue to be based at DataCite as he assumes this new role, according to the shared resourcing and governance model that ROR’s three operating organizations have adopted.

Joseph says, “I’m excited to dig in to the work of supporting this important open infrastructure. This means focusing my energy on scaling our core systems, ensuring data integrity for the registry, and maximizing interoperability to keep ROR at the heart of global scholarly communication.”

As we invite our community to join us in welcoming Joseph to the team, we also want to take this opportunity to highlight a few key technical projects that are underway this year and remind everyone what to look out for and how to get more involved.

Schema version changes

As we’ve shared in previous posts and community calls, ROR schema version 2 is now the default as of July 31 of this year, and we will officially deprecate schema version 1 in December 2025. If you have questions about what this means for you and/or how to update your integrations as a result of these changes, check out our documentation and get in touch with any questions via our technical forum or support@ror.org.

Implementation of API client IDs

In order to support and scale growing usage of the ROR API while also ensuring it can continue to remain open and free for everyone and anyone to use, we will be implementing new rate limits in early 2026 and enabling a client identification workflow for users to obtain higher rates. Learn more about what this means for you, and how to get started by obtaining a client ID.

Updates to affiliation matching

Affiliation matching is the number-one use of the ROR API, and we’ve been making ongoing tweaks over the years to refine our matching API to support various use cases as more and more systems integrate with ROR. Following development of a new matching strategy at Crossref, we are now working to adapt this for ROR to make our affiliation matching service even more useful and performant.

Join us

These projects and other technical activities have all been - and continue to be - shaped by active community input and collaboration. We take our position as community-driven open infrastructure very seriously, and that means working directly with users and our wider network of stakeholders to inform our development work, align our services to address concrete user needs, and deliver value at scale for our global community.

If you haven’t already gotten involved in our work, there are many ways to do so, including by joining our bimonthly community calls, by signing up for the ROR Technical Forum and/or ROR Community Forum, by suggesting registry additions and updates, and by submitting bug reports and feature requests.

Welcome, Joseph, and thanks to everyone for participating in this journey together!

In this interview with HighWire Press’s Tony Alves, we learn that thanks to customer requests and a PID-aware development process, the publishing platform DigiCore Pro uses ROR in form lookups and automatic extraction processes for author affiliations, funder identification, peer reviewer affiliations, user disambiguation, and research integrity.

Amanda French

Tell us about what made you decide to use ROR at HighWire, specifically in the DigiCorePro publishing platform.

Tony Alves

Sure. There were really a couple of different things. One is that we want to accommodate as many industry-accepted processes and tools as we can, and ROR is of course extremely important now. ROR is really helpful, because it provides us with a standardized list of institutions that people can pick from rather than typing them in. And because ROR is free and open, we could implement it without having to pay anybody, which is really, really helpful.

The second thing is that our customers are asking for it. We’re seeing ROR on Requests for Proposals (RFPs) as a requirement, especially now that it is being used at Crossref to identify funders. I like the fact that now there are fewer PIDs that we need to worry about. In those RFPs, people want ROR as not only an institution identifier for author affiliation, but they also want it as the funding PID. Also, of course, we can offer ROR to people who can’t afford a more expensive persistent identifier.

Amanda French

Great! You’ve touched on this a little bit, but how is ROR used in DigiCorePro?

Tony Alves

It is used as the basis of a pick list when people are selecting their institution so at that institution level it’s a typeahead, so you start typing, and you’ll get the ROR.

Amanda French

Do they see the ROR?

Tony Alves

They just see the institution name. They don’t see an identifier.

Amanda French

That’s actually what we recommend, so that’s great.

Tony Alves

We capture the identifier when the institution name is selected, and so when the metadata is transmitted, if the API wants the ROR, then that will be there for the affiliation institution field. Authors can have multiple affiliations, too, so you would then have multiple ROR IDs for the different affiliations. We have also implemented it as an identifier for funders.

Amanda French

That’s great.

Tony Alves

In both cases, we do try to anticipate the institution when we extract the metadata and match it with the ROR ID. We do that when we can for both funding and the person’s institutional affiliation. The person uploads their manuscript, and we then look through the manuscript for header information, pull that out, match it to the ROR ID, and present it as fields that are already filled in that they can change if need be to something that’s correct.

Amanda French

Oh, that’s interesting. It’s great that you do that. I think that’s a popular approach these days, and it’s so much easier for the author.

Tony Alves

We want to make it as simple as possible. The funder information is hard because there’s not a standard way of providing that information. People are still providing it in paragraph format somewhere, you know, even if you’ve asked for it in a certain place, right? So that’s always a challenge.

Amanda French

Those funding acknowledgements are very non-standardized. You mentioned that DigiCorePro transmits this metadata, including the ROR IDs, via an API. Where are some of the places that metadata might be transmitted to?

Tony Alves

Oh, there are all kinds of uses for our API. For example, it might go to a research integrity check tool, and depending on what data they want, the ROR and the affiliation information would be part of that. If there’s a place in their schema for ROR, it would fit in there. If their schema has just a general institution field, the institution name would go in there. There are also systems like Convey that collect conflict of interest information. Convey was created by the Association for American Medical Colleges. We query Convey’s database for conflict of interest information, so if ROR was part of that, it would be in that exchange.

Amanda French

I’ll contact Convey to see if they use ROR or are planning to use ROR!

Tony Alves

It also might be Prophy or Global Campus or Reviewer Credits, any of these reviewer finding tools that might get the ROR IDs from the API.

Amanda French

I know Prophy uses ROR quite heavily.

Tony Alves

There’s also ORCID. We can pull in the ROR if it’s in the ORCID record, or we can push the ROR out if somebody is establishing an ORCID record.

Amanda French

Yes! I actually can’t wait for the day when all of the organization and funder identifiers in ORCID are ROR IDs. And I think that day is not too far away. Increasingly, ORCID is standardizing on ROR.

Tony Alves

Anytime we need to push affiliation information and ROR is in the API, in the schema, that’s what we’ll do with the information.

Amanda French

What was the development process like for building ROR into DigiCorePro?

Tony Alves

We use an Agile process to develop the product itself. Part of what we do is to look at how something is implemented in BenchPress, then try to figure out what might be the best way to do that in DigiCorePro, which is more modularized. Specifically with ROR, we understand that there are multiple ways to use ROR, as I’ve been discussing, either as a person’s affiliation or as a funding ID. Also, there are multiple journeys for ROR: some systems are going to want the ROR IDs from us and some systems are going to send the ROR IDs to us. We create stories to describe how ROR will be used in DigiCorePro. How is the ROR ID going to get to us or be sent out? Where are we going to be using the ROR ID in the workflow process? Those get written into stories, and then those go into the development process as we develop the various features.

Persistent identifiers in general are always being considered for all the processes. Would persistent identifiers be needed in this process, in this interchange? What will those persistent identifiers be, or might there be a persistent identifier in the future that we might want to be sure is included in the design of this feature or function? We keep that in mind because we can see, just from our understanding of where the industry is going, that different aspects of the manuscript could potentially have different persistent identifiers.

For instance, for reviewers, it’s better to have the ROR for that person than it is to have the name of the institution, because the reviewer doesn’t get it right when they type it in. They’re just going to type in “UMass” or “Harvard.” We want to be able to be sure that if we are getting that information about that reviewer, that we have that consistent information.

It also helps us to disambiguate people. The ORCID is important for that, but sometimes when we give a journal staff person an opportunity to look at multiple people’s records, if somebody has somehow gotten registered two or three times, ROR is one of those things that will give them enough information to disambiguate and to identify which records need to be merged.

Similarly, with research integrity, if you have somebody who doesn’t have an institution that is in ROR, then potentially that may not be a real institution. We know that you don’t yet have all the institutions, necessarily, and that sometimes new institutions come up, but it can still be a useful piece of information.

So those are some of the ways that we think about ROR and other persistent identifiers in the development process. Will it be something that will be a useful element in this workflow? And what’s great is that because it’s open source, an open standard, we don’t have to constrain ourselves.

Amanda French

Is there anything else you want to say about ROR?

Tony Alves

I really appreciate the community around ROR, the vibrancy of that. I enjoyed PIDapalooza, and that was really a great launching pad for ROR because ROR is community-based and has an active advocacy group around it.

Amanda French

Lastly, any feature requests or suggestions for future directions?

Tony Alves

I think that from a systems perspective, being sure that your API is super-responsive in real time is really useful, because we can’t stop the submission process when an author is there to submit. I know it’s possible to download ROR and use it locally, but that means having a process to get the most recent version. That’s probably the safer implementation, because then if the API goes down, you’re not dependent on that, but then you don’t always have the most up-to-date information. That’s a struggle everybody has with these sorts of things when you’re dependent on an API for an essential process like identifying affiliation, so I think that’s an important part of your infrastructure to be sure is really stable.

Amanda French

Yes, ROR is quite a small dataset, so downloading the data and storing it locally is one of the implementation approaches we recommend, for precisely the reasons you mention. That being said, the ROR API has been very, very stable. It’s had great uptime.

However, because the ROR API is totally free and open, and because ROR is getting more and more popular, we’re seeing some excessive use that can cause issues. That’s one of the reasons why we recently started an initiative to implement client identification on the ROR API so that API users need to tell us who they are before receiving the current rate limit. We’re phasing that in gradually. Once it’s implemented, if somebody is using the ROR API incorrectly and therefore degrading the performance of the API for everybody else who’s using it, we can contact that API user and help troubleshoot the issue instead of just blocking their IP address.

Tony Alves

Good to hear.

Amanda French

Thank you so much for talking with me, Tony.

Tony Alves

Thanks so much.

We’re pleased to announce that Symplectic Elements now includes integration with the Research Organization Registry (ROR) - a key enhancement to our new External Organisations functionality and a significant step forward in enriching and disambiguating affiliation data across the research lifecycle.

What Are External Organisations in Elements?

In Symplectic Elements 6.22, released in early 2025, we introduced a new External Organisations data category. Symplectic Elements now maintains an authoritative list of External Organisations - institutions outside your own - which is used to normalize and enrich metadata relating to co-author affiliations, funding bodies, and other collaborative relationships. This dedicated data category is populated through a combination of trusted sources, including GRID, Dimensions, and now ROR.

This new approach enables more structured and consistent treatment of external entities across the system, supporting clearer collaboration insights, stronger reporting foundations, and more reliable metadata for open research initiatives.

This means:

• Fewer manual entries and less ambiguity around institutional names
• More accurate tracking of external affiliations across researchers, publications, and grants
• Improved support for reporting on collaborations and research partnerships
• A stronger foundation for funder acknowledgment metadata and impact analysis

How ROR Integration Supports Your Data

ROR is the leading open registry of persistent identifiers for research organizations. With ROR now integrated into Symplectic Elements, the platform can automatically enrich External Organisation records with up-to-date, authoritative metadata, sourced directly from ROR and from partners like Dimensions.

In addition to enhancing administrative workflows, these identifiers will increasingly become visible to researchers as affiliation data is harvested from publication sources such as Crossref - ensuring your users benefit from clearer, more consistent records.

This integration not only enhances the integrity of organizational data, but also supports institutions in tracking external partnerships, mapping research networks, and advancing open research infrastructure.

“Integrating ROR into our External Organisations model is aligned with our mission to connect institutions with high-quality data sources,” said Ben Heartland, Product Manager at Symplectic. “This enhancement ensures accurate and transparent organizational affiliation reporting for our community of researchers worldwide.”

By unifying metadata from across your institutional network with persistent, open identifiers, this development lays the groundwork for even richer research intelligence - all enabled through Symplectic Elements.

One of the highlights of 2024 for the ROR team and the ROR community was the release of version 2 of the ROR API and schema in April 2024. Before that milestone, we spent many months working together to develop an API and schema versioning policy: that policy states that “Plans to sunset a previous version will be announced at least 1 year prior to the planned sunset date.” We’ve been reminding the ROR community ever since the launch of v2 of the ROR API and schema that version 2 is the recommended, stable version and that v1 of the ROR API would be deprecated in 2025. Today, we’re sharing further details and more precise dates for the deprecation of version 1 of ROR so that ROR users have all the lead time they need to make any necessary changes.

Timeline for ROR v1 sunset

Here’s our basic timeline for deprecating version 1 of ROR.

What you can expect in July

Starting the week of Monday, July 28, 2025, ROR API requests with no version in the path will default to v2 instead of v1. What does that mean, exactly?

• If you use ROR API requests with v2 in the path, you do not need to make any changes.
• If you use ROR API requests with v1 in the path, you do not need to make any changes before July 28, but you should plan to make changes before December.
• If you use ROR API requests with no version in the path, you will need to change your API requests to include v1 in the path if you wish to continue retrieving v1 data. If instead you wish to start retrieving v2 data, you should make sure you understand the differences between ROR schema version 1.0 and ROR schema version 2.1 and update your script or application accordingly before you begin including v2 in the path of your ROR API requests.

Examples of ROR API requests

Here are three examples of the same request to the ROR API for a list of organizations whose type is “funder”: one that specifies version 2, one that specifies version 1, and one that does not specify a version.

ROR API request with v2 in the path

https://api.ror.org/v2/organizations?filter=types:funder

ROR API request with v1 in the path

https://api.ror.org/v1/organizations?filter=types:funder

ROR API request with no version in the path

https://api.ror.org/organizations?filter=types:funder

All three of these requests return the same number and set of records, but you’ll notice that the request without a version in the path currently returns records that use ROR schema version 1.0. As of the end of July, that will change, and an API request that does not specify a version in the path will return records that use ROR schema version 2.1.

Because ROR schema version 1.0 and ROR schema version 2.1 are significantly different, we recommend carefully evaluating the information you retrieve from version 1 of the ROR API before changing your API requests to version 2.

What you can expect in December

In early or mid-December 2025, we will sunset v1 of the ROR API and schema permanently. This means that ROR API requests with v1 in the path will cease to return responses, and we will no longer provide v1 files in the ROR data dump on Zenodo at https://doi.org/10.5281/zenodo.6347574. In addition, ROR documentation pertaining to v1 of the ROR schema and API will no longer be available.

Important additional points to note

Version 2 of the ROR API and schema was deliberately designed for both flexibility and stability, so we don’t anticipate any further breaking changes for quite some time. We’re always collecting feedback about future enhancements on our roadmap, so if there’s something you’d like to see ROR do, please let us know.

Lastly, and perhaps most importantly, we’re here to help! If you’ve got questions about how to update your script or application from v1 of ROR to v2 of ROR, please consult ROR’s documentation, join the ROR Technical Forum and post your question there, or email support@ror.org. The ROR Technical Forum is also a great place for developers who’ve successfully switched from v1 to v2 to share lessons learned with others, and we’d love to hear from you there.

After the sunset of the original version of ROR comes a night of restful sleep, and then a new day! See you bright and early on the other side.

The ROSA P digital repository, operated by the US National Transportation Library – part of the US Department of Transportation – is one of the top ten providers of ROR IDs for publisher identification and ROR IDs for funder identification in DataCite DOI metadata. In this short case study, Peyton Tvrdy tells us why and how her team did such exemplary work in producing metadata for open transportation research funded or produced by USDOT.

This article represents the first in an ongoing series of short case studies on ROR users that we’re calling “Four for ROR” in which we ask ROR users four questions about why and how they’re using ROR.

What made you decide to use ROR?

I was first introduced to ROR IDs Fall 2023 when I started at the National Transportation Library. A coworker of mine, Leighton Christiansen, was introduced to ROR and thought it would be a great idea to create ROR IDs for the Department of Transportation and our sub-organizations. That fall, I used the ROR submission form to create seven ROR IDs for the library. At this time we were unsure of how we would use them and integrate ROR into our repository.

While our repository does not have full ROR integration currently, we use ROR to identify our organizations in our DataCite DOI metadata. I decided to use the ROR API to match our organizations because our cataloging system has many different controlled values of the same organizations. ROR IDs help us to standardize our organization names while still giving proper attribution to organizations in our DataCite DOI metadata. Additionally, because I have a program that uses the DataCite API for DOI updates and maintenance, it was easy to integrate the ROR API for organization matching. Using ROR allows us to better standardize our organizations, fully utilize persistent identifiers, and create better metadata.

How do you use ROR?

Currently, the National Transportation Library only uses ROR IDs programmatically for our DOI metadata in my DOI parser program. ROR IDs are used for creator organizations, contributing organizations, publishing organizations, and funding organizations. However, ROR IDs are also used in the README files for our datasets in ROSA P. These README files contain important information about datasets, such as funding information, file information, licensing, and the preferred citation.

What were the steps you took to integrate ROR into your systems and workflows?

Cross-walking our organization names to ROR names and ROR IDs was a lengthy process, but it has resulted in better quality metadata. My program maintains a CSV of our organization names and their mapping to ROR. This allows my program to seamlessly match our organizations to organizations already matched to ROR. For organizations not in this spreadsheet, it queries the ROR API and has the user confirm or manually add a ROR match if there is one. That user response is then saved to the CSV for future use. This lessens the possibility for mistakes as each ROR ID and organization name are only matched once. This process is done for all organization fields in the DataCite schema. Mapping to ROR is especially easy for funder organizations because all of our repository’s research is funded exclusively by the Department of Transportation, so all of our contract and grant numbers are attributed properly to the Department of Transportation.

What else would you like to say about ROR?

ROR as an organization is an amazing resource for organizations looking to standardize and manage their organization names. The staff at ROR and the community calls are excellent resources to learn more about integrating organization persistent identifiers into your repository, documentation, or other processes. While matching controlled values to ROR IDs can be time-consuming, it is worth the effort in the pursuit of linked data.

In this dual case study, we learn why the Howard Hughes Medical Institute (HHMI) relies on OA.Report and why OA.Report relies on ROR to help HHMI track compliance with its open access policy.

Amanda French

Let’s start with introductions from you both. Joe, can you tell us your name, title, and organization?

Joe McArthur

I’m Joe McArthur. I’m the Executive Director of OA.Works.

Amanda French

Michele?

Michele Avissar-Whiting

I’m Michele Avissar-Whiting. I am Director of Open Science Strategy at HHMI, the Howard Hughes Medical Institute.

Amanda French

Wonderful. So, Joe, can you tell us about OA.Works and specifically about OA.Report as well?

Joe McArthur

OA.Works is a nonprofit project with the mission to build powerful, simple, open access tools with the vision of a just and kind information age. A lot of what we focus on day to day is building tools that make open access easy and equitable, and OA.Report is one of those tools. OA.Report is designed to help organizations unlock the research that they support, and we have a pretty strong focus on helping organizations implement open access policies. So we’ll do that by helping organizations find the articles that they’ve supported, whether that’s via funding or when their staff have authored it. We help them measure how open that research is, and then take concrete action to increase how much of it is open and increase things like policy compliance.

HHMI is a pretty typical example of our work. We help them track their research output using ROR and figure out how much of it is open access. It will help them figure out the policy compliance, but also other things, like how many articles have data availability statements. I know they’re using a lot of that data to then email authors to ask them to share their work publicly and raise open access policy compliance that way. I think that’s pretty cool. How we do this, I think, is quite neat. We use open data from big scholarly metadata sources like OpenAlex, and also lots of persistent identifiers like ROR, but we also layer in data the institution has that they may be gathering internally, data that only they have.

Amanda French

Do you mean the funder institution?

Joe McArthur

Funders, universities, they all have data that they’ve collected that can help with this task, and it can really help make your way through the open data effectively. And then we layer in on top of that data that we collect and clean in-house just to provide the broadest possible picture that we can for folks of what’s out there and give people the confidence to use that data to take action.

Amanda French

Fascinating. I don’t think I knew that it’s not only open data that you’re using.

Joe McArthur

It’s one of the most important parts of the work that we do. We’re massive fans of open data, and there’s so much you can do with it and persistent identifiers, but we do find, perhaps unsurprisingly, that if you enrich that open data with the data people are collecting internally, you can go further.

Amanda French

How do you collect that internal data from universities and funders?

Joe McArthur

During the setup process for OA.Report with institutions, we’ll talk them through the types of data that are often useful to us and work out how that’s going to be shared. The answer almost always is that essentially it will be open data by the by the end of the process. Normally, we work with them to identify what sorts of information they may have internally: that can be anything from previous grant reports stored in grant reporting systems or previous systems for tracking compliance or measuring output. It can be APC invoices, it can be lists of ORCIDs. It can be just names. It’s a whole suite of things, right? What I say is, “Just give that to us in whatever form you have it, and we will take it from there, clean it, do what we need to do, then use that to do output tracking on your behalf and help you figure out what isn’t open, what isn’t policy compliant.”

A classic example of this, and something that we work with a lot, is Grant IDs. We’ll have an initial set of Grant IDs from Crossref, which can, as we all know, unfortunately be incomplete or a bit messy. We will clean those up in-house. We will go out and collect more Grant IDs, and then from an organization’s internal data, we will match those Grant IDs to programs or initiatives inside the organization that may match to multiple Grant IDs or things like that. We’re not actually doing that in HHMI’s case, but that’s a good example of how we thread the needle between all these different data sources.

Amanda French

Fascinating. Michele, I’m going to turn to you now. First of all, just tell us about HHMI.

Michele Avissar-Whiting

HHMI is a research institute. We try to advance science by funding excellent researchers who are doing pioneering scientific research. We do it primarily not through grants: we’re not a traditional funder. We do have some grants, but the vast majority of our investment goes into actually employing researchers in the field at their institutions across the United States. That’s what the Investigator Program is. When you hear “an HHMI investigator,” that’s an investigator who is typically hosted at some other institution but has a dual affiliation with that institution and HHMI. We also have a research facility in Virginia called Janelia Research Campus.

This is relevant, I’m sure, to the conversation that we’re going to have next about how we use OA.Report, but it’s an important foregrounding. It’s an important aspect to how HHMI functions, because we fund people, not projects. We are finding individuals who are doing excellent work already and giving them the resources that they need to get to new breakthroughs in science.

Amanda French

Amazing. Tell us about why HHMI began to use OA.Report.

Michele Avissar-Whiting

On January 1, 2022 we launched our current open access policy, which is a requirement that all major research contributions that are produced by HHMI labs must be shared open access with the CC BY license immediately upon publication. It’s almost like the Nelson memo with the addition of the CC BY license requirement, but we did it earlier in 2022.

I came on about mid-2022 and when I came on, we still didn’t have a mechanism in place to enforce that policy beyond really manual work, just trying to pull in whatever publications we could find through things like PubMed and Lens.org and other databases to track compliance with the policy. So I started looking around. It was actually Ashley Farley at Gates who told me about Joe and OA.Report, saying that we should at least talk. I said, “Our situation is a little bit complex, because it’s not all outputs from HHMI, and not all outputs that are related to HHMI or have a grant from HHMI are subject to the policy. There are very specific criteria around it, so we would need to work with someone who could accommodate those finicky criteria.” And she said, “Just talk to Joe.” So I did, and that ended up being really the key to us being able to keep track of compliance with this policy.

We’ve been with OA.Works ever since. It was a process of probably about a year with a lot of back and forth to refine the specifics around what publications we need to be able to have visibility into and what information we need about them. We define “major contributions” as papers that have an HHMI lab head as the first, last or corresponding author, or a co-author in any of those positions. The first author is often a postdoc or graduate student who wrote the paper, and the last author is usually the principal investigator. The corresponding author is usually also the last author, but not always. So that’s very specific. If they’re a middle author, then that paper isn’t subject to the policy, and we don’t need to care about it for compliance purposes. That’s why OA.Report needs to be able to pick up on the order of authors, which before I spoke to Joe I didn’t know was possible to do. They have to be able to pull that out of the metadata and also to pick up on the metadata that ascribes corresponding authorship, which can be across multiple authors on a paper. So it’s complicated.

It’s also not every type of article type that’s subject to the policy. It’s only original research, so things like reviews and editorials and commentaries are excluded. There’s also a temporal aspect of it, which is becoming less important over time, but which was really annoying in the first year and a half or so that we were working together, which is that we only wanted to find papers that were submitted after the time that the policy went into effect. Submitted, not published, after January 1 2022. That was another piece of metadata that we needed, because if it was submitted on December 22 of 2021, then it’s not subject to the policy. We don’t want to be chasing people down for that.

Open access is a core tenet for us. It’s a massive part of our values. We fund this research so that it can help society, and we don’t think it can do that or be maximally impactful unless it’s free to read, obviously, but also maximally free to use. That’s where the CC BY license with no restrictions comes in. So Joe also needs to be able to pick up on CC BY versus CC BY-NC [non-commercial] versus CC BY-NC-SA [non-commercial, share-alike under the same license] – all of those derivations that are not actually compliant with our policy and that we need people to take action on to fix if they select the wrong license. And this has become the big challenge for us, because there’s a lot of confusion around licenses, and it’s really hard to communicate with authors why this license matters and how it’s being restrictive if you add these other letters to the end of the open license.

Amanda French

I remember the whole learning experience I went through about the difference between CC BY-NC and CC BY. I was opting into non-commercial uses for a long time, and then I realized through a great blog post by Bethany Nowviskie and some other discussion that actually that does cause problems.

Michele Avissar-Whiting

Right - on its face, it seems like maybe the better option, and you think CC BY-NC-ND [non-commercial, no derivative works] might be somewhat more protectionist, but still open. Then you learn that, for example, if you have an NC license, your work can’t be used in a textbook.

Amanda French

The other thing I wanted to mention here is that Crossref is planning a schema update, and one of the things that will be added is CRediT roles for contributors, which might help with that author order problem. It’s not going to put you out of business, Joe, but it’s something that I think a lot of people will appreciate.

Joe McArthur

I mean, I will say that one thing we’re a big fan of is being put out of business! Especially by big open metadata sources. Michele laid out all the challenges for, you know, figuring out whether or not the HHMI policy covers a particular article that may be associated with HHMI. This a problem that we face all the time, and it’s one of the most important pieces for us. What it means to measure policy compliance is to figure out, “Does the policy apply or not?” And it’s surprisingly complicated in a large number of cases, because 90% of the data that Michele mentioned is not available in open metadata in places like Crossref. Or it’s partially there, or it’s there in some sources but not others. So you’ve got to find other ways to get that data, and solving that is a big challenge that we take on at OA.Report.

But we’re really big fans of supporting the long term work that means we don’t have to do this. We don’t want to have to do this. We’d like to be supporting persistent identifiers and, you know, just data structures in the space that get us to not needing to do all of that leg work. We’ve been big proponents of Crossref’s Grant Linking System, and we’ve been working really hard with Crossref to try and speed up adoption of grant DOIs so that grants metadata is better and we don’t need to do so much work. It’s also great to hear that there are schema updates happening that will make it easier to spot the corresponding author, because it’s increasingly important for policies, and it’s really hard to get your hands on unless you are using multiple data sources or else you’re able to just go and read the paper yourself and pull out the corresponding author. That’s what we do now, but we’d prefer not to do it forever.

Michele Avissar-Whiting

And every journal has their own way of demarcating funding and authors. Another aspect of this that’s definitely relevant here is that what OA.Report is picking up on is not grants or Grant IDs or anything like that, and not even the word “HHMI” just appearing somewhere in the manuscript. It’s actually affiliations. That’s the flag that says to us, “This a paper that you might want to pay attention to.” It’s if somebody who’s the first, last, or corresponding author has the HHMI affiliation. And of course, there are name variants. There’s “HHMI” [https://ror.org/006w34k90], there’s “Howard Hughes Medical Institute” [https://ror.org/006w34k90], there’s “Janelia Research Campus” [https://ror.org/013sk6x84], which is part of HHMI. Sometimes people include HHMI with the Janelia Research Campus affiliation, but sometimes they don’t, and that gets into why ROR is necessary.

And sometimes people just don’t put that affiliation, and that’s a problem. That’s probably the next on the list of things for us to tackle with OA.Report, because right now that’s a blind spot for us. If someone just doesn’t put their HHMI affiliation, we don’t see it. I don’t know how often that happens, but it definitely happens with some frequency. So what we would like to do instead is pick up on ORCIDs, and of course those are not used universally, and so that’s another metadata problem. If we could always ensure that we are finding the right author on a paper, then we might not actually need the affiliation anymore.

Amanda French

I look at affiliation metadata in Crossref frequently, to see of course who’s sending ROR IDs to Crossref, but also to see who’s just sending those affiliations. And I’ll tell you, even for new content, I find it surprisingly low. It hovers around 60% of new records that have affiliations. [See ROR IDs in DOI and ORCID records for Crossref API queries that retrieve the number of records with affiliations since 2022 and the total number of records since 2022.]

Michele Avissar-Whiting

Joe, I think you use lots of other data sources, though, including OpenAlex.

Joe McArthur

Yes. The affiliations challenge speaks directly to why ROR is necessary. And yes, in the Crossref metadata, it would be great to see more publishers putting ROR IDs in directly, and I know the ROR team is hard at work nudging people to do that. So that’s awesome. And yes, one source that we work with a lot now is OpenAlex, and they will take that raw affiliation string from Crossref, match that to a ROR ID, and then everyone using OpenAlex gets the lovely benefits of ROR, even if the publisher is not putting them in. That’s one of the incredible things that open metadata and open sources gets us. Now, it would still be better if the publisher was putting the ROR in, but that’s one of the huge advantages of ROR as an open system. And, you know, increasingly, with community standards, people will go out and do that, and that’s really awesome.

There’s places where that’s still not far enough, though. One of the challenges that we see with HHMI data is that multiple affiliations are sometimes not handled fantastically by different systems. I know when we were doing the major author identification, I believe we were using OpenAlex data, Crossref data, PubMed Central data, and Europe PubMed Central data, because each of them was better at a particular slice of getting some of the corresponding, last, and first authors. Some of them were better at having all of the affiliations, but they’d have the authors in the wrong order, whereas some of them were better at having the right order of authors, but not all of the affiliations. It was a whirlwind! But towards the end, we got to a good place.

Michele Avissar-Whiting

I said that maybe if we had reliable ORCIDs for researchers we wouldn’t need the researcher affiliations, but actually it does need to be a sort of triangulation. We also get false positives when people wrongly put HHMI as their affiliation. They have a grant from HHMI, and they or their postdoc wrongly put HHMI as their affiliation. What we really need is some kind of source of truth that could pick up on the ORCID and pick up on the ROR and then see if it’s a part of the list that we have of people who have active appointments at HHMI. That would be the panacea.

Amanda French

I’m on a Committee on Publication Ethics (COPE) affiliations working group that’s addressing that to an extent. Sometimes people are claiming affiliations they don’t have, sometimes in order to get an APC deal, sometimes for prestige, but sometimes also just by mistake. The question came up of whether some kinds of fellowships should count as affiliations. There will be a paper coming out of that that will help at least to outline some definitions about what affiliation means and to make some recommendations about best practices.

Anyway, coming back to you, Joe. You’ve already talked a little bit about this, but why and how do you use ROR?

Joe McArthur

OA.Works has been using ROR as an organization identifier from the very start, since I think 2019. It was in another product just a little bit before OA.Report’s time. Our reason for that was because we wanted an open, community-owned ID that had good coverage globally and was supported by a good team. And you know, that still defines ROR today, except all of those things are more true. Within OA.Reports, ROR IDs are our primary IDs for organizations. We will use that to take a first pass at populating alternative names, relationships, things like that. And then we will use those ROR IDs to go and search other open data sources. If they support ROR, we will just directly go and look at those results, and that will be a lot of how we navigate the open data portion of our work. We’re often using multiple ROR IDs to do that, which as far as we’re concerned is a feature, not a bug. That’s a great thing about ROR. We’re also using the cross-links between between ROR IDs and other IDs, which are a useful starting point for us when we’re starting to get into the weeds with folks.

Amanda French

How did you first come to hear about ROR back in 2019?

Joe McArthur

I wish my memory was good enough to remember! We’ve been at this for over a decade now, and from day one we’ve been putting together open code sharing, open data, and generally being a community-run nonprofit, and so we are always on the lookout for friends. Somewhere around that time I remember that people were excited and talking about it.

And we would have been having conversations internally back then about, like, “How do we identify this organization? Do we come up with some ID internally that we are then shepherding? Do we write just the string, with all the obvious issues that come with that?” Even back then, the best option was to lean on a big, community-owned solution. And it’s been great to see ROR effectively become the standard, the clear way forward for identifying organizations.

Amanda French

I have seen that as well. And even among publishers, who have been slower to adopt ROR, I think they plan to do it: it’s just a question of doing the work. They’ve built all their workflows around other things, and it takes awhile to overhaul those. Just recently, I’ve been focusing a lot on scholarly association publishers, and they’re very open to it. I think over the next one to five years, we’re going to see a lot more ROR IDs coming from publishers in particular.

Joe McArthur

That’s great.

Amanda French

Wrapping up! Joe, do you have any feature requests for ROR, or things you hope we do in the future?

Joe McArthur

I don’t know if I’ve already given it away, but I mean, we are fans. We think ROR is terrific. We think it’s terrific today, and when we check in and see where ROR is headed, we’re excited. We’re excited to see ROR supersede Funder Registry IDs. We’re excited to see the API and the metadata curation models getting refined over time. And we’re excited to see the ROR team making lots of efforts to get global coverage of research organizations. No notes, in many ways.

I would say that an area we are especially supportive of is that as ROR becomes more and more of a standard, it’s important to make sure that it’s more and more representative of organizations, to make sure that we don’t miss any organizations out there that need a ROR. That global coverage is a thing that we think ROR is uniquely positioned to do, and we want to make sure that as we’re trying to support bibliodiversity and make sure that wherever research has been published, it can be found, discovered, used, that ROR is a big part of that. That’s our only “plus one,” really.

Amanda French

That’s a great point. The number of curation requests we’re getting for ROR is just a hockey stick curve upward, you know. We added a new curator in August who has been wonderful. We’re often focusing on metadata with non-Latin character languages, as well. We’re working on a long-term project to improve metadata for organizations with names in Chinese, for instance.

Joe McArthur

That’s really terrific. And that’s exactly the sort of things that we want ROR to be doing and that we’re really grateful for, because we at OA.Works are not going to become experts in Chinese-language organizations, but it’s important that the systems that get built around ROR can take advantage of that. Doing that work at ROR and doing that high level of curation across the dozens of languages that are important in the scholarly literature is just great. I’m really glad to hear that that’s being done.

Amanda French

Yes, that’s ongoing work here at ROR. Michele, what else would you like to say about your work, or about ROR, or about anything at all?

Michele Avissar-Whiting

I don’t know if it’s a question or a comment, but I’m looking at the item types for the ROR record for Howard Hughes Medical Institute, and the organization type is given as “Funder” and “Nonprofit,” which is true. But then, if I go to the record for Janelia Research Campus, which I see has HHMI as its parent, it’s listed as a “Facility.”

Now, the little nuance here is that HHMI, writ large, is a funder, but it’s primarily an institute, so we identify more as an institute than as a funder. What’s not captured in these item types is that the people who are employed by HHMI who are not at Janelia. We’re a little bit of a weird case, but I wonder whether it would make sense to also have HHMI as a “Facility.” As ROR starts to become better integrated into all these publishing systems, I would want Howard Hughes Medical Institute to be in that drop down as an affiliation. It wouldn’t be, necessarily, if it’s only categorized as a funder.

Amanda French

That’s a great note. The first thing I’ll say is that you can request that change if you like, or I can do it for you, and we can take a look at adding that type to the HHMI record. [Note: this change was made in January 2025, a few weeks after the interview took place.] The second thing I’ll say is that, as Joe mentioned, we’ve been doing a lot of work on evolving our schema and API, and one of the things that we’re probably going to look at fairly soon is revising that list of item types, because it isn’t very nuanced. We inherited the list of organization types from GRID, and there’s not very many of them.

The way ROR does this, is we put together a proposal, and then we ask for a lot of community feedback. Once we get to that point, I will be sure to send that to you, because actually edge cases like yours are super useful in making sure that everybody has what they need.

Michele Avissar-Whiting

I am happy to help or contribute to that conversation, however it’s helpful. I just wanted to flag that, because I wouldn’t want HHMI not to show up in a list of affiliations.

Amanda French

I think it would appear in a list of affiliations by virtue of the fact that it’s also listed as a “Nonprofit,” but that’s a good thing to flag. Actually, in version 1 of our schema, all the organizations only had one type, and it was only after we released version 2 that we introduced the “Funder” type and began adding it to records. That’s why most records that have that “Funder” type also have another type, such as “Government.” I’m sure HHMI was originally listed as a “Nonprofit” only, and then we added that “Funder” type to every record with a Funder ID mapping.

Michele Avissar-Whiting

It’s just one of those funny things. We’re always lumped into groups of funders, and, fair enough, we’re part of cOAlition S, and we are part of other funder groups. But even though we’re always considered by other people to be a funder, you’ll never hear somebody within HHMI refer to HHMI as a funder. It’s just interesting.

Amanda French

Anything else from you, Joe, about your work or about ROR?

Joe McArthur

I could say infinite things about both. No, just that we’re really grateful to the ROR team for five years of hard work now. We’re so grateful to the ROR team for providing an infrastructure that so many people can build on. And Michele, we’re grateful to HHMI for getting to work with you for over two years now.

Amanda French

Thank you both so much for sitting down with me.

Michele Avissar-Whiting

Happy to. Thanks.

Joe McArthur

Thank you.

This interview was conducted in November 2024.

Today, we’re delighted to let you know that Crossref members can now use ROR IDs to identify funders in any place where you currently use Funder IDs in your metadata. Funder IDs remain available, but this change allows publishers, service providers, and funders to streamline workflows and introduce efficiencies by using a single open identifier for both researcher affiliations and funding organizations.

As you probably know, the Research Organization Registry (ROR) is a global, community-led, carefully curated registry of open persistent identifiers for research organisations, including funding organisations. It’s a joint initiative led by the California Digital Library, Datacite and Crossref launched in 2019 that fulfills the long-standing need for an open organisation identifier.

In 2023, we shared our plan to transition the Open Funder Registry into ROR. More recently, we announced that we were planning to update our schema so that it is possible to collect ROR IDs where we currently collect Funder IDs such as in the funding metadata section for works and funder section for grants. Now that we have completed this work, Crossref members can start depositing ROR IDs where they would normally deposit Funder IDs. This update also means that the community, including funders, service providers, researchers, and data scientists can retrieve this metadata via our API.

So come and ROR with us and start depositing ROR IDs for both researcher affiliations and funding organisations.

Open Funder Registry-ROR transition

This is of course a significant first step in the Open Funder Registry to ROR transition.

We’ve always said that we would continue supporting Funder IDs in our schema and in our tools and services until the community is ready to transition - and we will. In the last year, Crossref and ROR conducted a series of Open Funder Registry user interviews to help us understand how it was being used and identify practical challenges to this transition in our members’ workflow (thank you to those who took part, it was incredibly useful!).

One major takeaway from this consultation was around the pivotal role that peer review management systems played in the Open Funder Registry-ROR transition. We look forward to seeing more service providers integrating with ROR in the future. If you are a service provider and are ready to integrate with ROR, drop support@ror.org an email.

Including ROR IDs in Crossref metadata

If you are ready to begin including ROR IDs in your funding metadata, you only need to include the ROR itself to identify a funder.

For example:

<fr:program name="fundref">
     <fr:assertion name="ror">https://ror.org/00fq5cm18</fr:assertion>
     <fr:assertion name="award_number">10.3030/725840</fr:assertion>
</fr:program>

Examples of more complex combinations of funding information are available in our documentation. This update has been made across all schema that support funding metadata.

Our grants schema has recently been updated to version 0.2.0 to support ROR IDs in place of funder identifiers as well. As with funding metadata, only the ROR ID needs to be supplied within the record:

<funding amount="750" currency="USD" funding-percentage="75" funding-type="APC">
  <ROR>https://ror.org/02twcfp32https://ror.org/02twcfp32</ROR>
  <funding-scheme>Sofa Lending Programme</funding-scheme>
</funding>

Although previously a funder name was collected with the funder identifier, for both grants records and funding data in an attempt to avoid redundant, incorrect or conflicting metadata, now we’re accepting an identifier only as the ROR ID has an existing metadata record. The organisation name exists within the record in the ROR registry and the ROR record is the authoritative source of the name.

ROR IDs in JSON outputs

We have an existing legacy practice of representing Open Funder Registry IDs as just a DOI, but ROR IDs are represented in the JSON outputs as a full URL with id-type “ROR”, for example:

Funding metadata

 "funder": [
      {
        "award": [
          "10.3030/725840"
        ],
        "id": [
          {
            "id": "https://ror.org/02twcfp32",
            "id-type": "ROR",
            "asserted-by": "publisher"
          }
        ]
      }
    ],

Grant funder information

"funding": [
              {
                "type": "infrastructure",
        "award-amount": {
                "amount": 750.0,
                 "currency": "USD",
                 "percentage": 75
            },
                "funder": {
                  "id": [
                    {
                      "id": "https://ror.org/02twcfp32",
                      "id-type": "ROR",
                      "asserted-by": "publisher"
                    }
              ]
            }
          }
        ]
      }
    ],

If you have any questions or feedback, get in touch with us at support@crossref.org!

It’s been six years since the initial launch of ROR, and our Annual Community Meeting has become a welcome tradition for the ROR community in every new year. It’s a chance for us to reflect on the past year, look ahead to the coming year, and express our gratitude for the diverse and growing community of research and information professionals who use and support ROR. Last year, more than 300 people registered for one of the ROR Annual Community Meeting sessions, and this year, that number grew to more than 500 registrants from 58 different countries, making this our most well-attended and most international Annual Community Meeting to date! If you weren’t one of those who registered or attended, here are some highlights.

Reflecting on 2024 and planning for 2025

ROR Director Maria Gould began this year’s “state of ROR” Community Update session by reflecting on the core values of openness and community that led to the creation of ROR and that continue to guide ROR today. She then gave a general overview of how these values are realized in ROR’s operations and governance.

ROR Product Lead Adam Buttrick and ROR Metadata Manager Riley Marsh reported on ROR data curation activities. In 2024, we added more than 4,000 new records to ROR and enhanced existing records with language tags and organization domains. Areas of focus for ROR’s data in 2025 include improving regional coverage for countries such as Japan and Portugal, adding new external identifiers to ROR records, and improving coverage of funders, scholarly societies, and publishers.

ROR Technical Lead Liz Krznarich reminded us that ROR reached a major milestone in 2024 with the launch of version 2 of the ROR schema and API. Four major technical projects are planned for 2025: improving the ROR API’s ability to match affiliation strings to ROR IDs, speeding up the overall response times of the API, implementing client identification, and sunsetting v1 of the ROR schema and API at the end of the year.

ROR Technical Community Manager Amanda French showcased some key ROR adoptions in 2024, including the American Physical Society and Web of Science, and mentioned that we’re looking forward to seeing ROR incorporated into Atypon and OJS 3.5 in 2025 as well as seeing increased use of ROR IDs to identify funders as well as author affiliations.

During the session, participants told us about their recent and forthcoming work with ROR, as well, and we discussed interesting questions such as “Which kinds of curation requests are the hardest to resolve?” We followed the Community Update with an informal, unrecorded drop-in session where we continued these and other discussions.

Learning how different countries use and encourage persistent identifiers

The National PID Policies and Practices panel included perspectives from the UK, the US, Czechia, Germany, Brazil, and Ireland. Rachael Kotarski, formerly with the British Library and now with the University of Chicago, mentioned that one key difference between the UK’s work to form a national persistent identifier strategy and the US’s nascent efforts to do the same is that the US is less focused on research assessment than the UK, so the key driving force in the US tends to be the minimum requirements by national funders for persistent identifier support in repositories.

Taras Hrendaš of the Office of the Government of the Czech Republic explained the methodology behind their annual evaluation of Czech research statistics, which they derive using a combination of data from their national CRIS system (IS VaVaI) and data from Web of Science. Institutional affiliation incoherence is a major problem for their data analysis, so the incorporation of ROR into the Web of Science will be a tremendous help for next year’s analysis.

Among the insights uncovered by Germany’s PID Network Deutschland, represented by Andreas Czerniak standing in for Steffi Genderjahn, is that the best-known PIDs are the DOI, ISBN, and ORCID, followed by GND, URN, and ROR. Andreas also announced the recent launch of the first version of Germany’s PID Monitor. Exciting!

Washington Segundo of Brazil’s IBICT gave an overview of six key components national research infrastructure in Brazil, including a national repository of theses and dissertations (BDTD), a portal to publication and data repositories (Oasisbr), the Brazilian current research information system (BrCris), an open repository and data lake that collects and processes FAIR data (Laguna), a directory of Brazilian electronic scientific journals (Miguilim), and a project to develop an open and non-centralized source of persistent identifiers (dARK). Despite technical, financial, and cultural challenges, IBICT is working to expand Open Science infrastructure in Brazil, disseminate the country’s scientific production, and reduce dependence on commercial platforms.

Finally, Michelle Doran of Ireland’s National Open Research Forum gave an overview of Ireland’s recommendations and roadmap for an Irish National PID Strategy, published in the fall of 2024. Immediate recommended actions include engaging with additional senior leaders across all sectors and maintaining engagement with international PID initiatives, while lessons learned include recruiting lots of willing volunteers to help promote the work and getting buy-in from funding bodies.

Focusing on the Asia-Pacific region

The session Successes and Opportunities for ROR in the Asia-Pacific Region was itself a definite success, and we plan to hold more APAC-focused sessions in APAC-friendly time slots in the future. Even though some clever folks in the Asia-Pacific region have already realized the value of ROR, there’s clearly lots of room to increase awareness and adoption of ROR in the Asia-Pacific region..

First, ROR Technical Community Manager Amanda French gave a brief introduction to ROR. Next, Aaron Ballagh of The Lens (based in Australia) explained how ROR is used to identify organizations in The Lens so that users can find scholarly works from researchers affiliated with a particular organization, then demonstrated an “institutional lineage” search lets a user find all the scholarly works associated with an organization and the organization’s “child” organizations.

Brietta Pike of the publishing arm of Australia’s National Science Agency, CSIRO Publishing, then outlined their “journey to capture and deposit better metadata” by taking steps toward ROR implementation as a “source of truth” for author affiliations. Scott Edmunds of GigaScience Press, based in Hong Kong, then recounted how the GigaByte Journal is “focused on the Interoperable in FAIR” to make their publications “AI-ready,” and completing the ROR integration for GigaScience’s journals and database is a key part of that.

Finally, Masatsura Igami of NISTEP described why and how they incorporated ROR into the NISTEP Dictionary of Organization names in order to better link this Japanese resource with overseas users. The NISTEP Dictionary “plays a central role in NISTEP’s efforts to compile and publish basic data on the research activities of Japanese universities and public institutions.” The presentation called for improved coverage of Japanese organizations in ROR (which is a priority for 2025) and concluded with a call for ROR to “connect the efforts being made in each country as a network.” A useful idea!

Join us in the coming months

ROR has plenty to keep us busy this year, and you surely do as well. Keep in touch with us by registering for ROR’s upcoming events, by signing up for the quarterly ROR newsletter, by joining the ROR Community Forum to receive more frequent announcements and participate in discussions, or by following ROR on Bluesky, Mastodon, or LinkedIn. Here’s hoping we all thrive in 2025.

OpenAlex has added a new metadata matching strategy co-developed by ROR and Crossref to its affiliation matching processes: ROR is also investigating the prospect of incorporating this new matching strategy into the ROR API in 2025.

If you’ve been reading our recent series of blog posts about metadata matching, you know that automatic metadata matching at scale is a topic dear to our hearts. Research metadata is full of unstructured and wildly variant text strings describing organizations, which makes connecting research outputs to research organizations unnecessarily difficult. Universities find it difficult to track what their affiliated researchers have produced; publishers find it difficult to associate author affiliations with subscribing institutions; and funders struggle to accurately track and assess the impact of their investments. Metadata matching can help alleviate these problems.

ROR and Crossref have been collaborating on extensive research into developing and evaluating new metadata matching strategies that have the potential to improve organization metadata throughout the scholarly communication ecosystem. One of the most promising of these new metadata matching strategies is one we refer to internally as “single search,” now being used by OpenAlex.

What is “single search”?

Single search is a new affiliation matching strategy developed in collaboration between ROR and the Data Science team at Crossref (which is currently hiring!). Whereas ROR’s existing affiliation matching makes use of multiple queries to a search index to return its results, the single search strategy is able to achieve approximately the same performance with just one query to said index. This allows the strategy to return matching results in a fraction of the time as the existing ROR affiliation matching while also using far fewer computing resources. The single search strategy is therefore particularly well suited for matching ROR IDs to affiliation strings in large datasets.

Partnering with OpenAlex

This past fall, we spoke with OpenAlex about our work on new strategies for affiliation matching, which resulted in a project to supplement their existing classification-based affiliation matching with the single search strategy. After testing against OpenAlex’s benchmark datasets resulted in the same high performance as our internal evaluations, OpenAlex was able to move forward with a full-scale implementation of this strategy across their entire dataset.

This implementation yielded impressive results: matching against the 140 million affiliation strings in OpenAlex created affiliations for 7,700 unique institutions, generating new affiliations for 1.2 million strings. This translated to new affiliations for 2.2 million works and added 4.5 million new author affiliations. The number of ROR IDs represented in OpenAlex also increased significantly, from 93,000 to 100,600, or approximately 90% of the active records in ROR.

Next steps

ROR is currently looking into how to bring the single search strategy to the ROR API in 2025, and Crossref will publish a dataset of all unique affiliation strings in the Crossref database matched to ROR IDs using the single search method, as well as a set of relationships between works, contributors, and organisations discovered through affiliation matching.

Our special thanks to OpenAlex’s Jason Priem and Justin Barrett for this collaboration. Stay tuned to all of our various communication channels and events for future updates on our work on metadata matching!

In our previous entry in this series, we explained that thorough evaluation is key to understanding a matching strategy’s performance. While evaluation is what allows us to assess the correctness of matching, choosing the best matching strategy is, unfortunately, not as simple as selecting the one that yields the best matches. Instead, these decisions usually depend on weighing multiple factors based on your particular circumstances. This is true not only for metadata matching, but for many technical choices that require navigating trade-offs. In this blog post, the last one in the metadata matching series, we outline a subjective set of criteria we would recommend you consider when making decisions about matching.

Openness

Matching tools come in many different shapes and sizes: web applications, APIs, command-line tools, sometimes even enchanted crystal balls showing matched identifiers emerging from a mysterious mist! No matter what form they take, an important consideration is whether the source code and all the related resources for the matching are openly available.

Matching strategies that are either closed-source, or rely on closed-source services for their matching logic, make it difficult to fully understand and explain matching processes. This lack of transparency also makes it impossible to adjust or improve the matching logic, since we cannot understand or improve code we cannot see.

Users are similarly impeded from identifying flaws or suggesting improvements to processes they are unable to examine. By blocking this community participation, we also lose the proven cycle of real-world testing, refinement, and validation that has strengthened myriad of open source projects. The cumulative impact of both minor and major community-driven refinements over time is incredibly valuable and should not be underestimated.

Using open source matching will also help build trust in the matching workflows and results. This is one reason why open source is one of the tenets of the Principles of Open Scholarly Infrastructure, adopted by Crossref, DataCite, ROR, and other organizations who build and maintain open scholarly infrastructure.

When evaluating matching strategies, we strongly recommend prioritizing those that are fully open source. This not only ensures their transparency and trustworthiness, but also allows for the kind of continuous improvement that results from this visibility and community engagement.

Explainability

In terms of our ability to understand and improve a matching strategy, using an open source model is only the first step. What typically matters most in the context of building and maintaining matching services is that we are able to understand their underlying code and have a clear model of how matches are derived from their corresponding inputs. Even if the matching code itself and all of the resources used in the matching are open, if they are poorly documented, lack reproducibility or tests, or are otherwise opaque, there is no guarantee that it will be possible to understand or improve the strategy. Striving for a high level of interpretability in our matching plays a determinative role in how well we can understand and modify our strategies in the future.

Being able to explain the behaviour of the matching will also help you to respond to and incorporate user feedback. When users encounter errors, you will be able to do things like advise them on how to modify or clean their inputs so that the results are better. Conversely, examining the behaviour of the strategy relative to user inputs and feedback can provide you with ideas for improving the matching.

Typically, heuristic-based strategies, such as those that use forms of search or string similarity measures, like edit distance, are easier to explain than, say, machine learning models. If a strategy uses machine learning, at least some internal decisions might be made by passing data through a complex network of algebraic equations. Those can be mysterious, non-deterministic, and are famous for being hard to interpret.

This doesn’t mean they should be avoided entirely – we have built and use many machine-learning based tools ourselves! Instead, it is a good idea to weigh how their inherent lack of explainability could affect your ability to continue work on the strategy and respond to user needs, relative to all the available options.

Complexity

Complexity is another aspect that can greatly affect how easy it is to maintain the strategy. Complexity is related to how many different components the strategy has and how difficult they are to use and maintain. When a strategy has multiple interconnected parts, each component becomes a potential failure point that requires discrete assessment and maintenance.

Consider, for example, two different approaches to a matching strategy: one that uses a single machine learning model versus another that uses an ensemble of models. A single model requires maintaining one set of training data, a single training pipeline, and one deployment process. If the model’s performance unexpectedly deteriorates, whether because of an issue with the training data, a configuration error, or the need for additional input sanitization, the source of the problem is easier to isolate and fix.

The ensemble, by contrast, combines multiple, specialized models, each requiring its own training data, tests, updates, and deployments. If one model in the ensemble is found to reduce the performance of the strategy, the interdependence between models can cause this degradation to cascade through the entire system and undermine its overall reliability. Correcting for these errors becomes more challenging. If fixing one model’s performance requires retraining or adjusting its outputs, this could require recalibrating the entire ensemble to maintain the balance between models, identify regressions, and prevent new errors from emerging.

In general, preferring simpler strategies not only reduces operational overhead, but also makes it easier to diagnose issues, test changes, and iterate on user feedback. When problems arise, having fewer moving parts means less places to look for the root cause and fewer components that could be affected by any fixes.

Flexibility

The metadata to which we match grows and changes over time. New records are created, existing ones are updated, with schemas changing and evolving alongside. The resources that underlie our matching are also not static. The libraries we depend on may deprecate features between versions or the taxonomies we used to categorize results might undergo significant revisions. We thus rarely have the luxury of deploying a matching strategy once and using it forever without any changes. A good strategy has to be flexible enough to adapt to such changes, with this adaptation also being both technically feasible and practical to implement.

Much of this flexibility is also determined by a matching strategy’s ability to incorporate new data. Strategies that use continuously updated databases or indices can immediately match against new metadata as it appears in the system. By contrast, some machine learning-based approaches require training on target matches and can thus be limited in flexibility and face more constraints. While some models can be incrementally updated to recognize new matches, others require retraining from scratch to incorporate these changes - a process that can be both time-consuming and resource-intensive.

Paying close attention to a strategy’s flexibility and favoring this aspect, when possible, can significantly impact its long-term viability. When comparing different matching strategies, flexibility should thus be a primary concern in your decision-making process.

Resources

Matching strategies can vary significantly in their resource requirements, including things like CPU and GPU utilization, memory consumption, storage capacity, and network bandwidth. These requirements are directly related to infrastructure costs and energy consumption, so when evaluating a matching strategy, it is necessary to assess its resource demands across all phases of the matching lifecycle. This includes things like initial model training, re-training, index construction, updates and management for all aspects of the strategy, as well as the real-world processing of matching requests. It is a good idea to measure and monitor resource usage carefully in considering which strategies to use, as the best performing strategy may also be too resource intensive to run as a service or might grow to this state over time with additional utilization.

Speed

Matching strategies can operate at a wide range of speeds, from milliseconds to minutes per match. Since the overall response time of a strategy can affect both system scalability and user experience, we should always assess the strategy’s performance for different usage scenarios and scales of data. While some strategies might perform adequately with small datasets, they can also exhibit exponential slowdowns as data volume and complexity increases or as concurrent requests grow in number. We should therefore consider carefully how requirements for matching speed might evolve with increased usage, data complexity, and total anticipated growth. The fastest matching strategy might not always be the best choice if it comes at the cost of reduced accuracy or requires large amounts of resources, but unacceptable latency can make an otherwise excellent strategy unusable in practice for many use cases.

Putting it all together

The typical life cycle of developing a metadata matching strategy is as follows:

1. Scoping: we define the matching task, along with its inputs and outputs.
2. Research: we research what existing strategies are available for our task and/or we develop our own.
3. Evaluation: we evaluate all available strategies, internally or externally-developed, exploring all of the aspects described above.
4. Decision: we choose which strategy (if any) we want to use in our production system.
5. Production setup: we prepare the production models, indexes, and other resources needed for the matching.
6. Maintenance: we monitor and adapt the strategy relative to changing data, user feedback, and new resource requirements.

In practice, these phases do not happen all at once, nor in this strict order. Often we need to proceed through multiple iterations of them to arrive at the best strategy. For example, if initial evaluation of a strategy yields poor results, we might return to the research phase to investigate other strategies or refine our understanding of the task. Often, during the maintenance phase, we receive feedback from users that indicates potential areas of improvement and then pursue them with a new round of research and evaluation.

As we cycle through these phases, ideally all the aspects described in this entry, along with the results of the evaluation, would be taken into account. Of course, this means that these decisions have to be based on multiple criteria and by making trade-offs between their performance and all other considerations. In making these complex and difficult choices, it is useful to consider two primary questions:

1. Are any of the considered matching strategies good enough for our use case?
2. Out of all the considered strategies that are sufficient for our use case, which would be the best?

The first question requires us to create clear and quantifiable criteria that allow for eliminating some of the potential strategies. As we have indicated, these could include things like the strategy being open source, minimum performance baselines using measures like precision or recall, and operational thresholds, like the strategy being able to return results quickly, relative to user expectations or the volume of data to be processed. It should be fairly easy to test these requirements and eliminate any strategies that fall short of them. If the strategies are difficult to assess, that is likely a mark against them.

If no strategies meet these criteria, we have two options: either to abandon matching entirely or to reassess and relax our criteria to align with the available options. While the former is always an option, adopting a more pragmatic lens, framing in terms of potential value (or harm) to the users, might be beneficial. Sometimes we approach matching tasks with too high expectations and a dose of realism helps us to re-center our perspectives. After more consideration, you might decide that your criteria were too stringent or realize that you need to better define and decompose the tasks to fit the available options.

When multiple strategies appear viable, the selection process becomes more nuanced. When evaluating strategies across these various dimensions, we should try to avoid placing undue weight on minor performance differences. Evaluation metrics are useful estimates of performance, but do not always translate to real-world applications and changing data. In cases where a more complex strategy offers only marginal improvements over a simpler alternative, the maintenance and operational benefits of the simpler solution often outweigh small performance gains.

This concludes our series on metadata matching, where we described the conceptual, product, and technical aspects of matching and its applications. We hope this overview was instructive and helps you to make better decisions about the use of matching in your own tools and services!

We are pleased to announce that Web of Science Core Collection™ now includes Research Organization Registry (ROR) identifiers. Users of the Web of Science™ can search for publications associated with an institution by using a ROR ID, and ROR IDs are available in organization profiles. In addition, Web of Science API Expanded now also exposes ROR IDs in query responses, making it easier for automated tools to retrieve standardized organization information and analyze research by institution.

“ROR is a trusted source of institutional identifiers and metadata that is being widely adopted across the scholarly communications landscape,” said ROR Director Maria Gould. “ROR’s open and carefully curated dataset and our robust and easy-to-use tools make it ideally suited for a range of implementations aimed at normalizing institutional data, and we’re pleased that ROR is being integrated into discovery services like the Web of Science to enrich institutional models and support key user needs.”

Tilla Edmunds, Director of Web of Science Content Management at Clarivate, said “We are pleased to integrate ROR IDs into the Web of Science Core Collection. This advancement highlights our commitment to strong partnerships with the academic community, working together to support and progress scholarly communication globally.”

Clarivate™ has long supported ROR as a community-led solution to the pressing challenge in scholarly publishing of variant affiliation names for the same organizations. The new ROR features in the Web of Science, developed in response to customer feedback, are only the first phase of ROR adoption for Clarivate. Now that more and more academic publishers are beginning to use ROR, Web of Science plans in the future to collect ROR IDs for researchers’ affiliations directly from publisher metadata, thus lessening and perhaps one day even eliminating the problem of nonstandard research affiliation information.

Learn more about this ROR integration from the Web of Science release notes and a presentation given at the September 2024 ROR Community Call by Tilla Edmunds, Director of Web of Science Content Management at Clarivate.

As 2024 comes to a close, we’re reflecting on where ROR is now, and the answer is unmistakable: ROR is just about everywhere. We continue to be thrilled by the extent to which ROR is recognized as the obvious choice for identifying research and funding organizations, as evidenced by a wide range of new ROR integrations, by our extremely busy curation queue with over 8,000 requests this year to add new organizations and update existing ROR records, by the bustling traffic of 14 million monthly requests to the ROR API, by the fact that the number of downloads of our open dataset has nearly tripled in the past year, by the honor of being named a finalist for the 2024 ALPSP Innovation in Publishing Award, and by continual additions of enthusiastic ROR supporters to our very active and engaged community.

Here are some highlights from ROR’s banner year in 2024.

ROR’s schema and API evolve with the release of v2

April of 2024 marked a major milestone with the release of ROR API and Schema 2.0 – an official departure from the schema ROR initially inherited from GRID. Version 2 added features such as created and last-modified dates on all records, a place to store language codes for all names, and the addition of a domains field. All of these changes were shaped by extensive community feedback, and we’re pleased that more than 40% of ROR API users have already switched to version 2.

After community feedback indicated the need for additional location information, the ROR team implemented a minor, non-breaking release of the ROR schema in December of 2024. Schema 2.1 makes location information in ROR more granular and adds continent names and codes to the location information available.

Publishers, repositories, and knowledge graphs adopt ROR

ROR adoption continues to increase among many systems – and many kinds of systems. We began the year with Springer Nature’s January account of their internal use of ROR at the ROR Annual Community Meeting, and then at mid-year came the announcement of the American Physical Society’s adoption of ROR both internally and as an affiliation identifier in Crossref DOI metadata. Drawing publishers’ attention to the strategic importance of ROR in open scholarly metadata, Crossref and CWTS Leiden worked together to add metrics for ROR IDs and author affiliations to Crossref’s Participation Reports.

ROR has already seen tremendous uptake among repository systems, and this year thanks to 4Science, both DSpace 8.0 and DSpace-CRIS now support ROR IDs for both author affiliations and funders.

Scholarly Knowledge Graphs (SKGs), too, showed their awareness of ROR’s strategic value in 2024. OpenAlex continues to be a key ROR adopter, and 2024 saw the launch of a new tool for community-based correction of OpenAlex’s automatic matches of text affiliation strings to ROR IDs: the Works Magnet, developed in conjunction with France’s Ministère de l’Enseignement Supérieur et de la Recherche (MESR). Finally, Clarivate™ gave the ROR Community a special preview of their planned integration of ROR into the Web of Science at the ROR Community Call in September and launched the integration only a few weeks later.

Too many other systems incorporated ROR this year to list here, but we want to emphasize our appreciation for every new ROR integration as well as the improvements to existing ROR integrations that we’re beginning to see from many early adopters of ROR, including ORCID. We’re always happy to feature ROR users on our integrations list, in case studies on our blog, and as presenters at our bi-monthly community calls.

Use of ROR as a funder identifier continues to grow

ROR has been increasing its coverage of funders by reconciling ROR with the Open Funder Registry since well before the announcement in late 2023 that the Crossref Open Funder Registry will eventually be deprecated in favor of ROR. With the release of v2.0 in April, ROR added a “funder” item type to indicate which ROR records are mapped to the Open Funder Registry to assist with this use case. We’ve also published a guide to help users transition from Open Funder Registry to ROR.

Beginning in late 2023 and throughout 2024, we’ve seen many widely used scholarly systems begin to use ROR IDs to identify and disambiguate funders. CHORUS Dashboard & Reporting Services, OA.Report, ChronosHub, and OpenAlex are all among the systems that either use ROR exclusively for this purpose or else map other funder identifiers to ROR IDs.

Repository systems have been particularly quick to adopt ROR for funder identification, and nearly 450,000 DataCite DOI records registered by these systems currently include ROR IDs for funders. See our blog post Using ROR for Funder Identification for more details about how repository systems such as Dryad, Zenodo, and Figshare use ROR for the funder identification use case.

ROR expands global coverage of research organizations and scales curation capacity

We pride ourselves at ROR on providing a registry with truly global coverage that takes an international perspective on research organizations and information systems. In 2024, we undertook several initiatives aimed at improving regional coverage of organizations, while at the same time experiencing organic growth from Mexico, France, and Indonesia. We also called for community feedback on a proposal to add additional external identifier mappings to ROR records, including regional identifiers such as the European PIC code, which many ROR users have said they would find useful. In 2025, we plan to complete our evaluation of potential new external identifiers and then add them to ROR records.

More generally, we have processed over 8,000 curation requests in 2024, a 44% increase over 2023, adding a total of 4,098 new records to the registry this year and updating 106,944 records. Recognizing the importance of scaling our capacity to respond to curation requests quickly while continuing to produce accurate and complete metadata, this year we not only revamped and improved our curation tools and codebase but have also added a new team member: Riley Marsh, Metadata Manager.

With the release of v2, ROR records now also include a place to store a language code in all name fields and a new field for an organization’s top-level domains (e.g., madeira.gov.pt instead of https://www.madeira.gov.pt), and in 2024 we carefully populated these new, empty elements with accurate values for thousands of records, further enriching ROR’s metadata and making it of even greater value to the global research information community.

ROR’s community activities highlight broad engagement

ROR’s community activities in 2024 included publishing pieces on various topics of interest. For instance, ROR and Crossref collaborated on a blog post series explaining various aspects of automatic metadata matching that proved very popular and has even been adapted into French by Carole Melzac of ABES. Merci, Carole! We also published additional case studies with ROR users and a guide to RRID and ROR for facilities.

ROR also participated in and co-organized several global events both in person and online, notably a webinar co-organized with 4Science on New Features for Organization Information in DSpace with ROR, a session at PIDFest on Curation and Community: Setting Metadata Policies at ROR, and an ACRL Choice / LibTech Insights webinar in collaboration with OpenAlex on Open Affiliation Metadata: How Recent Innovations Help Libraries and Librarians.

And, of course, as always, we held our regular bi-monthly online Community Calls featuring demos and presentations by adopters as well as engaging discussion about future directions. If you’re making a New Year’s resolution to get more involved with ROR in 2025, we encourage you to join the ROR Community Forum (now 376 members strong!) to receive notifications of upcoming Community Calls as well as other news and event announcements. Save the dates, too, for ROR’s Annual Community Meeting on February 4th and 5th, 2025 – registration is open now for four terrific sessions over the course of two days.

We give our heartfelt thanks to all of you who have supported ROR this year by sharing your thoughts, asking us questions, doing the work at your own organizations, making financial contributions, and in a hundred other ways demonstrating that you understand the value of ROR not just for your own benefit, but for the benefit of the entire research information ecosystem. We look forward to collaborating with you in 2025.

This blog post explores the difference between “core facilities” in RRID and “facilities” in ROR and provides guidance for those who run facilities on how to effectively use these identifiers.

What is RRID and what is its scope?

RRIDs (Research Resource Identifiers) help identify a wide variety of resources which are inputs to experiments, especially biomedical experiments. Examples of resources identified by RRID:

• Antibodies
• BioSamples
• Cell lines
• Core facilities
• Databases
• Instruments (capital equipment)
• Organisms
• Plasmids
• Reagents
• Software

RRIDs started in 2014 as an agreement between 25 journal editors to improve how research resources, especially antibodies, are cited in the scientific literature. The infrastructure has been supported in the FAIR Data Informatics Lab at the University of California at San Diego and SciCrunch Inc, a company devoted to improving scientific literature. RRIDs.org has recently become a stand-alone not-for-profit entity, enabling sustainability. The RRID registry at scicrunch.org currently includes nearly 25,000 records, including around 3,000 facilities.

What is ROR and what is its scope?

ROR (Research Organization Registry) IDs help identify research organizations, defined as “any organization that is involved in research,” including organizations that produce, fund, facilitate, manage, and publish research as well as organizations that educate or employ researchers. Examples of research organizations identified by ROR:

• Archives
• Colleges and universities
• Companies that fund or conduct research
• Government agencies and units that fund or conduct research
• Hospitals and healthcare centers
• Laboratories
• Nonprofits and non-governmental organizations that fund or conduct research
• Private foundations that fund research
• Research facilities
• Research institutes
• Research libraries
• Scholarly publishers
• Zoos

ROR is an initiative jointly supported and managed by Crossref, DataCite, and the California Digital Library that makes it easy to disambiguate institution names and connect research organizations to researchers and research outputs. ROR was launched in 2019 after three years of consultations with working groups and stakeholders, developed to solve the problem of identifying organizations in an open and community-driven way. The ROR registry currently includes nearly 110,000 active records and is currently widely adopted by many essential scholarly systems.

Definitions of facilities for RRID and ROR

As you can see above, RRID includes a resource type for “core facilities,” and ROR includes an organization type “facility.” Despite this apparent similarity, the definitions for “core facilities” in RRID and “facilities” in ROR differ quite a bit, and indeed there is very little overlap between core facilities in RRID and facilities in ROR. Let’s look now at how RRID and ROR define “core facility” and “facility” and at some examples of each.

What are “core facilities” in RRID?

Core facilities are centralized resources within universities that offer access to instruments, technologies, and expert services. These semi-autonomous units are essential for scientific and clinical investigators, enabling them to conduct cutting-edge research without the need for individual investment in expensive equipment or staff.

Core facilities have joined forces and created a society called the Association of Biomolecular Resource Facilities (ABRF), which began as a society for US biomedical facilities but now encompasses international facilities from many disciplines. The ABRF Core Marketplace is the listing of member Core Facilities that is primarily used to advertise Core services. One issue that has come up for Cores is the lack of proper citation of the facilities and the tremendous waste of time that searching the literature takes from each Core.

The ABRF Core Marketplace joined the RRID initiative as a partner and now lists RRIDs on each active Core listing, gathering data about Core usage from the RRID efforts, search of the PubMedCentral database as well as the Core directors. “RRIDs allow our cores to be uniquely identified and many of our members push their users to use RRIDs to cite / acknowledge their facilities” explains Nate Herzog, the Core Marketplace director. As of this writing, the ABRF Core Marketplace maintains 1597 active facility profiles, 576 of which have been cited by RRID.

Some active core facilities with RRIDs include the following:

• New York University School of Medicine Langone Health Microscopy Laboratory Core Facility (RRID:SCR_017934)
• Salk Institute Razavi Newman Integrative Genomics and Bioinformatics Core Facility (IGC) (RRID:SCR_014842)
• Stanford University Vincent Coates Foundation Mass Spectrometry Laboratory Core Facility (RRID:SCR_017801)

These core facilities with RRIDs do not have corresponding ROR IDs, but the parent organizations do:

• New York University - https://ror.org/0190ak572
• Salk Institute for Biological Studies - https://ror.org/03xez1567
• Stanford University - https://ror.org/00f54p054

What are “facilities” in ROR?

ROR’s list of organization types includes a value for “facility,” defined as “A specialized facility where research takes place, such as a laboratory or telescope or dedicated research area.” There are currently over 11,000 organizations of the type “facility” in ROR.

Facilities in ROR include the following:

• The Square Kilometre Array Telescope, Australia - https://ror.org/00ydnew86
• Medical Research Council (MRC) Prion Unit, University College London - https://ror.org/043j90n04
• Stanford Linear Accelerator Center (SLAC) National Accelerator Laboratory - https://ror.org/05gzmn429

These facilities in ROR do not have corresponding RRIDs, reflecting the broader nature of the definition of “facility” in ROR as well as ROR’s history of comprehensive coverage of non-US and non-biomedical organizations. Facilities in ROR are usually organizations like national laboratories that may be associated with a university but are not dependent on it.

What are the main differences between “core facilities” in RRID and “facilities” in ROR?

RRID identifies core facilities because they can be considered as “inputs to experiments,” especially when the main contribution of the core facility is to provide researchers with access to particular instruments. University-based core facilities are typically funded by individual grants, and in many cases those are instrument-related grants intended to bring a capability to a certain university. These are typically large instruments that are made available to multiple individual laboratories in the department or university. The success of the core facility depends on the usage of the core resources, the publication of scientific work based on the data produced at the core facility, and on the successful interactions between core facility staff and local investigators. Because these facilities are grant-driven, an accounting of impact typically centers on counting papers that core facilities / instruments have been acknowledged in. RRIDs help core facilities better identify the manuscripts that need to be accounted for as they measure their impact.

In the majority of cases, these core facilities are not in scope for ROR because they are subsections or service offerings at a university instead of being functionally separate organizations in the way that hospitals, national laboratories and facilities, and large research institutes are. The national or international facilities that have ROR IDs are also acknowledged by researchers in the way described above, and they may well also be core facilities, but they are independent organizations that do not rely on a single parent institution or on individual grants. These independent facilities may also employ researchers who cite that organization as their primary affiliation. One of the primary uses of ROR is to help organizations track research outputs by contributor affiliation.

The difference between facility acknowledgement and contributor affiliation can be seen by looking at a sample article published in 2022 in the journal Leukemia by Baeten et al. The authors are affiliated with research organizations that can be identified by ROR IDs, including The University of Chicago Medicine Comprehensive Cancer Center (https://ror.org/042wftp98).

In the same paper, the authors also acknowledge support from the DNA Sequencing and Genotyping Facility Core, also known as the University of Chicago Functional Genomics Core Facility, identified by RRID:SCR_019196.

In this example, then, RRIDs help track core facility acknowledgements, while ROR IDs help track contributor affiliations. These are the typical use cases for RRIDs and ROR IDs. That being said, ROR IDs can also be used to help track facility acknowledgements if the facility has a ROR ID.

Let’s look now at how to use RRID and ROR.

How to use RRID and ROR

How to use RRIDs

RRIDs are usually added to the main text of the manuscript; they are present in materials or resource tables (STAR Table example), in the methods section text (see example), or in the acknowledgement section (many core facilities are cited this way). This enables the core owner, and any robot (or undergraduate student) to easily extract the fact that this manuscript is associated with this resource or core facility. Being part of the JATS standard means that publishers will pull these identifiers into article metadata for you, at some point, but we are not holding our breath until that happens. Based on the hard work of robots and undergraduate students, RRID pages on both the RRID and Core Marketplace sides list the papers associated with each Core Facility, owners of facilities can also help to fill out this data, for example https://n2t.net/RRID:SCR_019195.

ORCID records also accept RRIDs, enabling data to be harvested from or sent to an individual researcher’s ORCID profile, which means that analysts can examine ORCID data for RRIDs. RRIDs are proposed for inclusion in the next DataCite metadata schema, which means that one day analysts will also be able to look for RRIDs in DataCite data.

How to use ROR

ROR is “infrastructure for infrastructure,” meaning that ROR is meant to be directly incorporated into software applications or added to scholarly metadata by information professionals rather than being used by individual researchers or by facility administrators. ROR offers both a REST API and a downloadable dataset for use by developers in their applications, and those who register DOIs with Crossref or DataCite can also include ROR IDs in DOI metadata. Typically, a ROR-enabled system will build a ROR-powered lookup into forms so that a user can simply search for the name of their organization and select it without ever seeing or using the ROR ID. Research outputs such as journal articles and datasets can then be tracked by all kinds of different ROR-enabled systems, including OpenAlex, the Web of Science, Crossref, and DataCite Commons.

ROR is the preferred organization identifier for contributor affiliations in ORCID, DataCite, and Crossref and is used in DOI metadata for many item types, including journal articles, datasets, projects, instruments, software, data management plans, and grants.

Recommendations for Facilities

If you are managing a facility, it’s important to adopt a strategy that incorporates either a ROR ID, an RRID, or both. Here are our recommendations.

1. University-based core facilities: Obtain an RRID. RRIDs will enhance the visibility of your research and allow for proper attribution in research outputs. To request an RRID simply go to the RRID website and fill out a form (3 fields are required for the initial step). You will direct the curator to look at your core facility webpages and we will in turn ask you to also register with CoreMarketplace (one click option is coming soon!). Curation will provide a set of instructions for how to use your RRID and we will start tracking papers citing your core as soon as the record is approved.
2. Other facilities: Obtain a ROR ID. If your facility has multiple parent organizations or is frequently used as an affiliation to identify research outputs (e.g., in contributor affiliations or dataset attributions) without reference to its parent organizations, first check to see if your facility is already in ROR, and if not, request a ROR ID. A ROR ID enables disambiguation from your parent organization(s) while preserving the connection, allowing for proper attribution and impact tracking.
3. Educate researchers about RRID: Inform researchers about the importance of using RRIDs and encourage them to include RRIDs in their publications for accurate representation of the resources used. Your acceptance letter will include the pointers from a survey from cores that implemented RRIDs successfully.
4. Choose systems that use ROR: Check to see if the tools and platforms used within your facility to manage and track your activities use ROR, and if not, ask the provider to integrate ROR (see more in the FAIR Facilities Report).

Conclusion

The use of persistent identifier systems like RRID and ROR is essential for the effective management and recognition of research facilities. Implementing these identifiers will not only streamline operations but also foster collaboration and transparency in research. Using these identifiers can help facilities of various kinds enhance their visibility, track their impact more effectively, and contribute to the larger academic community.

In this interview with Curvenote cofounder Rowan Cockett, we envision a world in which an authoring and publication platform helps scientists collaborate earlier, publish faster, and easily use structured metadata to create fully connected and highly interactive publications and portfolios.

Amanda French

Thanks for speaking with us. Tell us about Curvenote.

Rowan Cockett

Curvenote’s mission is to free science from static PDF documents to enable researchers to continuously share more interactive, reproducible, and richly-linked scientific content. Curvenote develops authoring and publishing tools for researchers, societies, and institutes, with a focus on computational research. We build authoring tools, including a “What You See Is What You Get” (WYSIWYG) editor, with the goal of making both the way that science is authored and the way that science is published a lot easier. We’re also thinking about how scientific metadata is included, trying to make sure that scientific publications are born with metadata all the way through.

We’ve done some really exciting integrations with ROR, as well as with ORCIDs and DOIs and RRIDs. For instance, one thing we’re thinking about is that as authors are writing, instead of having a citation manager where you’re copying the name and the title and the year into your text as a text-only citation, you just copy in the persistent identifier and then it automatically queries all of the scholarly databases against that citation for you. And then you get a very beautiful site that you can publish out of that. I use the word “site” rather than “journal article,” because I think one of the things that is difficult for scientists to do is to have a portfolio of their work. When you’re using the publishing ecosystem, your work is scattered all around the place. It’s really difficult to show all of the papers and articles and blog posts all in one collection on a landing page on your website.

We’re also elevating a lot of scientific computational work by letting people show off interactive Jupyter Notebooks. I think by bringing all this a little bit closer together and into the scientist’s control, you can experiment a lot more with how you’re presenting your work. You can push on things like reproducibility and interactive apps that are educational rather than focusing only on novelty in the scientific paper sense. What we’re trying to make possible is the ability to continually update that project, that paper, that article, that blog post, and to make sure there’s good metadata throughout that entire process such that it’s actually integrated into the scholarly ecosystem.

Amanda French

Would it be fair to say that Curvenote is a digital-first platform?

Rowan Cockett

Oh, yes. It creates beautiful PDF documents, but that is not the point.

Amanda French

Does this come out of your own scientific training?

Rowan Cockett

Yes. I have a background in Geoscience. I started creating interactive visualization tools for my peers in undergrad and understood that the web is just an exciting platform on which to disseminate these things. I saw my peers tinkering with geologic models and seeing the results right away, and there was this really close interactivity loop where they could discover things just by using those models. That was something that I learned in undergrad. I started and then sold a company around that over the next few years. Then I was working at that acquiring company for about four years, working in large-scale version control system collaboration for environmental science, civil engineering, and mining companies, and creating platforms for those companies. I was in charge of a large team over there.

While I was doing that, I was also doing my PhD in Computational Geophysics, so lots of open source technology, lots of Python, and lots of Jupyter. We were deploying courses that use those interactively in the classroom, and we took those on lecture tours. So we saw how powerful it was to put research tools directly in the hands of students all over the world, and rather than giving them a PDF with a picture of something, you give them a direct connection to the computation. With a PDF, there are no avenues back to where this came from. The actual scientific work behind that PDF document is all just lost. That was a really eye-opening experience of integrating reproducible science and open source communities into the medium that you’re using to communicate.

And while I’m doing that, I’m also going through scientific publishing and, like, getting my PDF paper rejected from multiple journals. Just honestly the worst.

Amanda French

And how long did it take to get those rejections?

Rowan Cockett

I was running a 65-person team at the same time as I was trying to finish my PhD, and I was in spurts of trying to get my PhD over the line, so it took three years for me to publish that paper. And the most disappointing thing is that at the end of those three years, someone reached out with a dataset that could use my work. That was the first time that had happened, and we completely missed out on the collaboration possibilities. If I had published right away and got that out there, I could have collaborated with that person at the start and made a much bigger impact.

My research would have been much more applicable with a field data set. I was doing theoretical modeling, and I did the first 3d model and inversion of a vadose zone dataset of infiltrating water in an agricultural setting. And that was all theoretical and computational: I demonstrated that this was possible, but not in a field sense. I didn’t actually go and apply it anywhere. But there are people who are collecting data that I could work with, and because of the scientific publishing ecosystem, I had zero capability to work with those people, and I think it limited the impact of that research project. By the time the research was published, I was no longer working on that project, and I think that’s just a shame, honestly.

Amanda French

And tell us about how MyST Markdown fits into this.

Rowan Cockett

I started Curvenote with the idea that you need that direct connection between research and publishing — working and sharing. That experience I described taught me that you need to be able to get your work out there and you need to get feedback right away. It needs to be born on the web with metadata right from the start. I think most scientists do their writing in Word documents, so that was the experience that we were going after for Curvenote: a What You See Is What You Get text editor that integrates with computation and metadata in a much richer way.

With the computation aspects, I think what I underestimated was that the communities who work in Python and Jupyter are much more used to text markup than to working in WYSIWYG editors. There are ways to work across those spectrums, but to enable that reproducibility, we needed to move into that text markup space. So we joined the Executable Books project, we open sourced all of Curvenote’s command line tools, and that was the start of the MyST-MD project, which is growing and helping the Jupyter Book ecosystem to create tutorials and educational materials. The MyST-MD project has recently become part of the Jupyter Books project officially, which is really exciting. Jupyter won the Open Science Award from the White House in the spring, and it’s making significant Open Science progress. We are bringing science communication tools to the core of Jupyter and integrating those metadata tools into the fabric of the Jupyter ecosystem.

Amanda French

That’s fascinating. I didn’t realize all the connections to Jupyter.

Rowan Cockett

Yeah, we’ve been working very hard.

Amanda French

How does Curvenote address the concept of peer review?

Rowan Cockett

There are different ways to publish MyST sites. We have a publishing system for Curvenote, so you can publish something yourself and just post it (for example to GitHub pages), but if you want a DOI on that, if you want to incorporate it into a scientific journal and have some of these better integrations into the scholarly system that are archival and that involve peer review, for example, that needs a system around it. And yes, the Curvenote publishing system and platform enables the creation of peer-reviewed journals. We’re working with AGU, the American Geophysical Union, which is a large-scale publisher. They’re thinking about how to do these web-first papers, these computational articles, and they’re experimenting with that and running that on the Curvenote platform. And that’s the same thing we’re doing with the Microscopy Society of America, which was just presented in Nature Magazine.

Amanda French

Got it.

Rowan Cockett

The North Carolina School of Science and Mathematics is also using Curvenote – that’s high school students learning about ORCIDs, RORs, and DOIs for the first time! There’s a teacher there who wanted his students to be able to have a breath of fresh air in the science ecosystem and to learn what it would be like for them to publish in six years. And that means they’re publishing immediately. They’re getting DOIs, they have ORCIDs, they have a ROR! Their work is born open access, and it’s born with reproducibility, and it’s born with the datasets integrated. It’s just so cool. Their journal is called The Morganton Scientific.

Amanda French

Very cool! There’s so many people experimenting right now with new publishing models, new peer review models, new models of open preprint dissemination, that kind of thing. And it’s so clear that it’s a generational type of thing. I don’t think younger people coming into science have the patience for the kind of print-first model that persists, which has just been replaced with a PDF in place of print.

Rowan Cockett

Yes, exactly. And that’s the way that we’re framing Curvenote, as a next-generation tool, a platform where scientific computation, scientific reproducibility, and scientific metadata are deeply integrated. And what does “next generation” mean? It means that it’s easy enough for a high school to get up and running and host their own journal! They can do that by themselves, and it’s open access, and it’s better than most journals out there. I think that it’s just exciting. Eighteen-year-olds can do this, so there’s no excuse!

Amanda French

Tell us about how you’re using ROR now and how you might intend to use it in the future.

Rowan Cockett

There are a couple of different ways. There are the ROR IDs in affiliation metadata that we’re collecting, so inside of Curvenote, you can add your ROR ID in the metadata, and that’s collected along with your affiliation. And then we have just started integrating that into the MyST ecosystem, so now, when you paste a link to ROR, you can hover over that anywhere in your document, and you can see the integrated metadata.

And again, this represents the ideal of a scholarly ecosystem that is rich with APIs and structured data, and our authoring tools need to be able to show the value of that as you’re working. You see a paper, you copy the DOI, and then you don’t have to think about the references, you don’t have to think about APA formatting. You see a ROR, you copy it into your document, and then it turns into the name of the organization with all of the information directly available in a hover reference.

Amanda French

Cool. And do you have a model for funding information in Curvenote?

Rowan Cockett

We rely on the same funding-statement standards that we have developed into the MyST ecosystem. Part of the work we’ve been doing with the Sloan Foundation and with AGU was collecting the funding information and making sure that that’s easy to author in a YAML-based format. That’s something that is presentable to authors, so that they can write their funding information in with a couple of DOIs. We focused on the Crossref Funder ID as the main integration point there.

Amanda French

Right. That is currently the standard for a funder identifier. I don’t know whether you saw that Crossref did announce last year that over the long term ROR will be the standard for identifying funders as well as other research organizations. We’re in the process of doing a lot of curation in ROR to make sure that all the funders are represented, and Crossref is working to make sure that ROR IDs can be accepted as funder identifiers in the Crossref schema. So just be aware that that’s coming. [See our Funder Registry transition guide for more information.]

Rowan Cockett

We will make those changes. That’s good to know.

Amanda French

MyST is a flavor of Markdown, correct? How is it different from regular Markdown?

Rowan Cockett

Yes, it is a flavor of Markdown. We’re pretty loose in the way that we talk about the MyST ecosystem, because there’s really three parts of it. First, there’s the flavor of Markdown – the actual syntax that you write and see. Second, there is the structure of that after it has been parsed. That piece is much more similar to JATS – there’s basically a one to one mapping of that next-generation scholarly structured data to what is in JATS XML today. There’s a MyST spec describing the structure of these documents, which we’re working on in the MyST community. Third, there are the command line tools, which make it very, very easy to get up and going. So MyST is more than the markup language: again, you paste a DOI in there and it pulls in the references, it talks to various external services, and MyST brings that authoring experience fully together.

Amanda French

Great. Thanks for that.

Rowan Cockett

Currently, there’s 15% adoption of the MyST language in the scientific documentation community, and that’s actually quite large.

Amanda French

That’s quite good, yes!

Rowan Cockett

That’s active documentation for projects, and there’s a couple of caveats in there, but that’s 15% right now. We’re working with the Python community to bring them over into this MyST ecosystem. That’s one of the entry points to a large part of the scientific computation ecosystem: teaching people how to author in this markup language, providing metadata in a structured way, and then coming all the way back. Over the coming years, MyST will be a default language in the Jupyter ecosystem that’s available. We’re starting to integrate MyST into the fabric of how scientists are authoring computational documents and computational articles.

Amanda French

It’s very smart to connect to things like Jupyter – tools that people use all the time. At one point, Jupyter was probably itself just a tool that nobody knew about, just used by a few different people. And then it was everywhere, and it became something everyone was using all the time, because it filled a need.

Rowan Cockett

Yeah. There’s a great TED talk called “How to Start a Movement.” It’s a dance party. The take-home there is that you need to join the things that are already existing and put your effort behind them. It takes a different kind of leadership to join these movements rather than start your own thing.

This is very much our approach, to reinforce existing communities, to bring new experiences to life inside of those and be that strengthening and connecting tissue. I think that’s somewhat lacking in how scientists experience the scholarly ecosystem in their day to day work. They have DOIs, they use Google Scholar, they see the list of papers that cite them, but they don’t have that direct connection to adding metadata in the way that they should. It should be tangible. They should get a hit of dopamine.

Amanda French

The UI you’ve made is actually really fabulous, because you can see the metadata, right? You can see all the connections underneath. People who don’t code or people who don’t really work with a lot of metadata often have a hard time understanding the importance of persistent identifiers. I think it helps when people can see, visually, that the identifier is connected to this knot of information.

Rowan Cockett

Right.

Amanda French

How was ROR to integrate, technically? Was it difficult?

Rowan Cockett

It took less than 45 minutes, and then we were up and going.

Amanda French

Amazing. That’s great to hear. What else would you like to say about ROR or about your work?

Rowan Cockett

The next few years in scientific publishing are going to look very different. I think there’s a wave that’s been building up over this time. We’ve been working for five years on Curvenote, MyST and the underlying authoring tools, and the scientific publishing ecosystem and the players in that space don’t see that coming, because they’re not really paying attention to educational tutorials that people are publishing by themselves, or blog posts that are using this, or scientific documentation.

This is where people are doing their work. That’s where people work every single day. And if we are extending the publishing ecosystem out from that, and deeply integrating it into the fabric of how people are working daily, that is a very different future than how we think about scientific publishing today. It’s much more about science communication. How are you getting feedback as fast as possible? How are you directing micro-expertise? How are you thinking about how to integrate with services like PREreview to bring that attention closer to when you’re doing a statistical analysis? You can get your figure out in front of people. You can get your methods section out in front of people.

And that’s a possibility that is being enabled by these various communities working together. That is a radically different reimagining of how the science infrastructure is supporting the work and communication of scientists. It can help support today’s publishing infrastructure, but it doesn’t have to just be that. We can work and share and do that as fast as possible and that can be supported by the scientific and scholarly infrastructure as well. And that again includes DOIs, and ORCID, and ROR, and things like that. I think it’s an exciting vision, and it’s happening.

Amanda French

Thanks so much for speaking with us.

Rowan Cockett

Thank you.

In our previous blog post in this series, we explained why no metadata matching strategy can return perfect results. Thankfully, however, this does not mean that it’s impossible to know anything about the quality of matching. Indeed, we can (and should!) measure how close (or far) we are from achieving perfection with our matching. Read on to learn how this can be done!

How about we start with a quiz? Imagine a database of scholarly metadata that needs to be enriched with identifiers, such as ORCIDs or ROR IDs. Hopefully, by this point in our series this is recognizable as a classic matching problem. In searching for a solution, you identify an externally-developed matching tool that makes one of the below claims. Which of the following would demonstrate satisfactory performance?

1. It is a cutting-edge, state-of-the-art, intelligent-as-they-come, bullet-proof technology! All the big players are using it. You won’t find anything better!
2. The tool was tested on the metadata of 10 articles we authored, and many identifiers were matched.
3. The quality of our matching is 98%.

Okay, okay, trick question. The correct answer here is to opt for secret answer #4: “I wouldn’t be satisfied by any of these claims!” Let’s dig in a bit more to why this is the correct response.

The importance of the evaluation

Before we decide to integrate a matching strategy, it is important to understand as much as possible about how it will perform. Whether it is used in a semi or fully automated fashion, metadata matching will result in the creation of new relationships between things like works, authors, funding sources, and institutions. Those relationships will then, in turn, be used by the consumers of this metadata to guide their understanding and perhaps even to make important decisions about those same entities. As organisations providing scholarly infrastructure, we must therefore take it as our paramount responsibility to understand any caveats or shortcomings of the scholarly metadata we make available, including that resulting from matching.

Proper evaluation is what allows us to do this, as it is impossible to know how well a given matching strategy will perform in its absence. This is true no matter how simple or complex a matching strategy may seem. Complex methods can be tailored to data with specific characteristics and might fail when faced with something different from this. Simple methods might be only appropriate for clean metadata or a narrow set of use cases.

Beyond complexity, matching strategies themselves vary widely in character, inheriting biases from their design, training data, or how a problem has been formulated. Some prioritise avoiding false negatives, while others focus on minimising false positives. Even a generally high-performing strategy might not be perfectly aligned with your specific needs or data. In some cases, the task also itself might be too challenging, or the available metadata too noisy, for any matching strategy to perform adequately.

Evaluation is, again, how we understand these nuances and make informed decisions about whether to implement matching or avoid it altogether. By now, it should also be clear that the notion “we don’t need to evaluate” is far from ideal! Given its importance, let’s explore how evaluation is actually done.

Evaluation process

In general, a proper evaluation procedure should follow the following steps:

1. Preparation of an evaluation dataset containing many examples of matching inputs and the corresponding expected outputs.
2. Applying the strategy to all inputs from the dataset and recording the responses.
3. Comparing the expected outputs with the outputs from the strategy.
4. Converting the results of the above comparison into evaluation metrics.

From this accounting, we can see that there are two primary components for the evaluation process: an evaluation dataset and metrics.

Evaluation dataset

It’s useful to conceive an evaluation dataset as the specification for an ideal matching strategy, describing what would be returned from our forever-elusive perfect matching. When creating such a dataset, what this means in practice is that it should contain a number of real-world, example inputs, along with the corresponding ideal or expected outputs, and that all data should be in the same format as the strategy is expected to process. The outputs should themselves also confirm the strategy’s overall requirements, for example, by being consistent with its cardinality, meaning whether zero, one, or multiple matches should be returned and under what circumstances. In terms of size, it’s generally useful to calculate the ideal number of evaluation examples using a sample size calculator or using standardised measures, but as a quick rule of thumb: less than 100 examples is probably insufficient, more than 1,000 or 2,000 is generally acceptable.

It is also important that the evaluation dataset be representative of the data to be matched in order to ensure reliable results. Using unrepresentative data, even if convenient, can lead to biassed or misleading evaluations. For example, if matching affiliations from various journals, building an evaluation dataset solely from one journal that already assigns ROR IDs to authors’ affiliations might be tempting. The data, having been already annotated, allow us to avoid the tedious work of labelling, and we might even know that it is produced by a high-quality source. This is still, unfortunately, a flawed approach. In practice, such datasets are unlikely to represent the entire range of affiliations to be matched, potentially leading to a significant discrepancy between the evaluated quality and the actual performance of the matching strategy, when applied to the full dataset. To assess a matching strategy’s effectiveness, we have to resist shortcuts and instead do our best to create truly representative evaluation datasets to be confident that we’ve accurately measured their performance.

Evaluation metrics

Evaluation metrics are what allow us to summarise the results of the evaluation into a single number. Metrics give us a quick way to get an estimation of how close the strategy was to achieving perfect results. They are also useful if we want to compare different strategies with each other or decide whether the strategy is sufficient for our use case, removing the need to compare countless evaluation examples from different strategies against one another.

The simplest metric is accuracy, which can be calculated as the fraction of the dataset examples that were matched correctly. While a commonsense benchmark, accuracy can be misleading, and we generally do not recommend using it. To understand why, let’s consider the following small dataset and the responses from two strategies:

Both strategies achieved the same accuracy, 0.67, making one mistake each on the second affiliation string. However, a closer examination reveals that these error types are distinct. The first strategy matched to an incorrect identifier, while the second refused to return any value illustrating the limitation of accuracy as a measure: it generally fails to capture important nuances in strategy behaviour. In our example, the first strategy appears more permissive, returning matches even in unclear circumstances, while the second is more conservative, withholding them when uncertain. Although using such a small dataset would preclude drawing any definitive conclusions, it highlights how relying on accuracy alone can obscure differences in performance.

For evaluating matching strategies, we instead recommend using two metrics: precision and recall. To recap from our previous blog post:

• Precision is calculated as the number of correctly matched relationships resulting from a strategy, divided by the total number of matched relationships. It can also be interpreted as the probability that a match is correct. Low precision indicates a high rate of false positives, which are incorrect relationships created by the strategy.
• Recall is calculated as the number of correctly matched relationships resulting from a strategy, divided by the number of true (expected) relationships. It can also be interpreted as the probability that a true (correct) relationship will be created by the strategy. Low recall means a high rate of false negatives, which are relationships that should have been created by the strategy but were not made.

Applying these measures to our prior example, the strategies achieved the following results:

• Strategy 1: accuracy 0.67, precision 0.5, recall 0.5
• Strategy 2: accuracy 0.67, precision 1.0, recall 0.5

As we can see, while both strategies have the same accuracy, using precision and recall better describes the difference between the two sets of results. Strategy 1’s lower precision indicates it made false positive matches, while Strategy 2’s perfect precision shows that it made none. The identical recall scores show both identified half of the possible matches.

Of course, results calculated using such a small dataset are not very meaningful. If we obtained these scores from a large, representative evaluation dataset, it would indicate to us that Strategy 1 risks introducing many incorrect relationships, while Strategy 2 would be unlikely to do so. In both cases, we would still expect approximately half of the possible relationships to be missing from the strategies’ outputs.

Which one is more important to prioritise, precision or recall? It depends on the use case. As a general rule, if you want to use the strategy in a fully automated way, without any form of manual review or correction of the results, we recommend paying more attention to precision. Privileging precision will allow you to better control the number of incorrect relationships added to your data. If you want to use the strategy in a semi-automated fashion, where there is a manual examination of and a chance to correct the results, pay more attention to recall. Doing so will guarantee that enough options are presented during the manual review stage and fewer relationships will be missed as a result.

To get a more balanced estimation of performance, we can also consider both precision and recall at the same time using a measure called F-score. F-score combines precision and recall into a single number, with variable weight given to either aspect. There are three commonly used types, each calculated as the weighted harmonic mean of precision and recall:

• F0.5: Precision is weighted more heavily. It can be understood as a score that is 50% more sensitive to precision than recall. A high F0.5 score indicates a measure of performance that minimises false positives.
• F1: Equal weight is given to both precision and recall. It can be interpreted as the most balanced score in this set. High F1 indicates good overall performance, with both false positives and false negatives being minimised equally.
• F2: Recall is weighted more heavily. It can be understood as a score that is 50% more sensitive to recall than precision. A high F2 score indicates a measure of performance where false negatives are minimised.

Each of these variants allows for fine-tuning the evaluation metric to align with your expectations for a specific matching task. Choose whichever reflects the relative importance of precision versus recall for your use case.

To summarise, to avoid falling prey to misleading sales pitches or silly quizzes, it is important to have a good understanding of the performance of any strategies you are building or integrating. With thorough evaluation, including a representative dataset and carefully considered metrics, we can estimate the quality of matching and, by extension, its resulting relationships.

Now that we’ve covered how to evaluate effectively, we can move on to some other aspects of metadata matching. Our next blog post will take a final, more holistic view of matching, exploring some complementary considerations to all of the preceding. Stay tuned for more!

We are thrilled to introduce a new member of the ROR pride: Riley Marsh joined the ROR team in August as our new Metadata Manager.

As Metadata Manager, Riley will support the day-to-day activities of curating the ROR registry, including triaging incoming requests and processing changes for new registry releases. Riley will work closely with ROR’s Curation Advisory Board to maintain ROR’s metadata policies and practices as well as working to understand and support various community use cases that might influence ROR’s curation processes and procedures. Based at Crossref, which is one of ROR’s three operating organizations, Riley will also contribute to metadata schema development activities on the Crossref side.

Riley brings a strong background to this work, most recently having worked at Tulane University Libraries in the Digital Scholarship & Initiatives department, supporting digital collections, electronic theses and dissertations, and various digitization projects across campus and beyond.

Former Metadata Curation Lead Adam Buttrick continues to be involved in ROR in his new role as persistent identifiers product manager at California Digital Library.

With Riley joining the team, it is an opportunity to reflect on and emphasize the importance of the workflows, policies, infrastructure, and above all the people that ensure ROR’s metadata is carefully curated and actively maintained so that our global user base has access to clean, comprehensive, and accurate records.

As a reminder, anyone can suggest registry additions and updates via this form. All suggestions are processed according to ROR’s curation policies, which are developed with our Curation Advisory Board, and tracked openly on GitHub so you can see what has been suggested and follow suggestions as they go through the process. Registry updates are published approximately once a month, and the latest data is available on Zenodo and in the ROR API.

For more information about curation, read more on the registry web page and in the ror-updates GitHub repository.

Since ROR’s launch in 2019, the ROR API has been provided at no cost and with no registration or credentials required. While requiring no registration or identification reduces barriers to entry, it makes managing the health and stability of the API challenging amid constant and growing use.

In recent months, “impolite” use of the ROR API has presented a particular challenge, which causes degraded performance for all API users. Examples of “impolite” use include:

• Sustained high levels of requests just below the rate limit from multiple coordinated IP addresses
• Use of proxies and frequent IP changing to evade rate-limiting

IP-based rate limiting, which ROR already has in place at a limit of 2,000 requests in every 5 minutes, is not an effective solution to the behaviors listed above. Additionally, monitoring and blocking impolite behavior based on IP address is not a scalable solution as ROR API use continues to grow. ROR is therefore seeking to implement a lightweight solution to allow monitoring and managing API traffic based on criteria other than IP address.

Goals

• Maintain stability and usability of ROR API by enabling throttling/blocking based on some sort of client identification rather than IP address

• Allow ROR team to better support users by enabling ROR team to contact users whose requests generate lots of errors, are incorrectly formatted, etc and provide them with assistance fixing their requests

• Allow ROR team more insight into API usage patterns than can be gathered from IP addresses alone, which help to guide technical decisions and architecture approaches

Scope

ROR is seeking feedback on the the following proposed change:

The potential addition of API client identification in order to receive a rate limit of 2000 requests per 5 minute period. Requests without identification will receive a lower rate limit of 50 requests per 5 minute period.

Any additional revisions to ROR’s API are out of scope for this proposal, but can be submitted for consideration to ROR’s roadmap.

Proposed changes

In keeping with ROR’s objective of maintaining low barriers to entry, two options for identifying API clients have been developed. Both approaches involve providing an additional URL parameter or request header with each API request. Neither of these approaches are intended to provide authentication or authorization, therefore values provided are not secret and can be passed and stored as plain text. Full details of the proposed changes, pros and cons, privacy considerations, examples, and implementation timing are available in the proposal draft.

Giving feedback

To give feedback, please visit the proposal document and suggest changes to the text or add comments in the margin by October 4th, 2024. Feedback about the “Questions to consider” noted in the document is particularly appreciated. We’ll give a summary of the responses and discuss implementation at the ROR Community Call in November.

Let us know what you think! We appreciate your time and all that you do to help us make ROR the best service it can be.

In our previous installments of the blog series about matching (see part 1 and part 2), we explained what metadata matching is, why it is important and described its basic terminology. In this entry, we will discuss a few common beliefs about metadata matching that are often encountered when interacting with users, developers, integrators, and other stakeholders. Spoiler alert: we are calling them myths because these beliefs are not true! Read on to learn why.

If you have stuck with us this far in our series, hopefully, you are at least a bit excited about the possibility of creating new relationships between the works, authors, institutions, preprints, datasets, and myriad other objects in our existing scholarly metadata. Who would not want all of these to be better connected?

We have to pause for a moment and be honest with you: metadata matching is a complex problem, and doing it correctly requires significant effort. What is worse, even if we do everything right, our matching won’t be perfect. This may be counterintuitive. Perhaps you’ve heard that matching is not a hard problem, or have encountered people surprised that a matching strategy returned a wrong or incomplete answer. Sometimes, it is obvious to a person from looking at some specific example that a match should (or should not) have been made, so they naturally assume that a change to account for this has to be simple.

Misconceptions like these can be problematic. They create confusion around matching, drive users’ expectations to unreasonable levels, and make people drastically underestimate the effort needed to build and integrate matching strategies. So let’s dive right in and debunk a few common myths about metadata matching.

Myth #1: A metadata matching strategy should be 100% correct

Anyone who has built or supported a matching strategy has likely encountered the following belief: it is possible to develop a perfect strategy, meaning one that always returns the correct results, no matter the inputs. The unfortunate truth is that while one’s aim should always be to design matching strategies that return correct results, once we move beyond the simplest class of problems or artificially clean data, no strategy can achieve this outcome. In thinking through why this is the case, some inherent constraints become obvious:

The inputs to matching are often strings in human-readable formats, which can vary wildly in their structure, order and completeness. Since they’re intended to be parsed by people, instead of machines, they’re inherently lossy and frequently unstructured, anticipating that a person can infer from the source context what is being referenced. Matching strategies, although built to make sense of unstructured data, unfortunately, don’t have the luxury of this flexibility. A strategy has to account for translating a messy, partial, or inconsistent input into a correct and structured match.

Consider, for example, the following inputs to an affiliation matching strategy:

1. "Department of Radiology, St. Mary's Hospital, London W2 1NY, UK"
2. "Saint Mary's Hospital, Manchester University NHS Foundation Trust"
3. "St. Mary's Medical Center, San Francisco, CA"
4. "St Mary's Hosp., Dublin"
5. "St Mary's Hospital Imperial College Healthcare NHS Trust"
6. "聖マリア病院"

In order to correctly identify the organisations mentioned here, the matching strategy must be able to distinguish between different ways of representing the same institution, disambiguate multiple institutions that have similar names, and handle variant forms for the parts of each name (Saint/St./St), identify the same name in different languages (“聖マリア病院” is Japanese for “St. Mary’s Hospital”), and make assumptions about partial or ambiguous locations translating to more precise references. While a person reviewing each of these strings might be able to accomplish these tasks, even here there are some challenges. Does “St Mary’s Hosp., Dublin” refer to the hospital in Ireland or a separate hospital in one of the many cities that share this name? Should we presume that because “聖マリア病院” is in Japanese, this refers to a hospital in Japan? Would someone, by default, be aware that St. Mary’s Hospital in London is part of the Imperial College Healthcare NHS Trust, such that inputs one and five refer to the same organisation?

An additional challenge lies in the quality of the data, which in the context of matching, encompasses both the input and the dataset being matched against. In real world circumstances, no dataset is fully accurate, complete, or current and certainly not all three. As a result, there will always be functionally random differences between inputs to the strategy and the entities to be matched. A theoretically perfect matching strategy would thus need to distinguish between inconsequential discrepancies resulting from gaps, errors, and variable forms of reference and actual, meaningful differences indicating an incorrect match. As one might imagine, this would require near total knowledge of the meaning and context for all inputs and outputs, a nigh-on impossible task for any person or system!

As a consequence, no metadata matching strategy will ever be perfect. It is unreasonable for us to expect them to be. This does not mean, of course, that all strategies are equally flawed or destined to forever return middling results. Some are better than others and we can improve them over time. Which brings us to the next myth:

Myth #2: It is always a good idea to adapt the matching strategy to a specific input

Matching strategies are not static. They can – and should – be improved. There is, however, a deceptive trap that one can fall into when attempting to improve a matching strategy. Whenever we encounter an incorrect or missing result for a specific input, we treat this problem like a software bug and try to adapt the strategy to work better for it, without considering all other cases.

The more complicated reality is that the quality of matching results is controlled through a complex set of trade-offs between precision and recall that determine the kind and number of relationships created between items:

• Precision is calculated as the number of correctly matched relationships resulting from a strategy, divided by the total number of matched relationships. It can also be interpreted as the probability that a match is correct. Low precision indicates a high rate of false positives, which are incorrect relationships created by the strategy.
• Recall is calculated as the number of correctly matched relationships resulting from a strategy, divided by the number of true (expected) relationships. It can also be interpreted as the probability that a true (correct) relationship will be created by the strategy. Low recall means a high rate of false negatives, which are relationships that should have been created by the strategy but were not made.

The tradeoff between precision and recall roughly means that modifying the strategy to improve recall will decrease precision, and vice versa.

Imagine, for example, we received a report about a relationship that was missed by matching because of a partial, noisy, or ambiguous input. We might be tempted to resolve this issue by relaxing our matching criteria. Unfortunately, this will have a cost of a higher overall rate of false positive matches.

Conversely, if we encounter a case where the matching has returned an incorrect match, we might attempt to make the matching strategy stricter to avoid this result. We should remember, however, that this may have the consequence of causing the strategy to skip many perfectly valid matches.

Striking this balance becomes even more difficult when attempting to address multiple issues at once, or considering constraints like the time and resources consumed by each aspect of the strategy. Each choice can compound the individual effects in unanticipated and expensive ways. The aim of matching ultimately then can’t be to achieve perfect results for every single case. Fixing one particular situation might not be desirable, as it can result in breaking multiple other cases. Instead, we have to find a locally optimal balance that optimises the strategy’s utility, relative to these inherent limitations. This means accepting some level of imperfection as not just inevitable, but necessary for implementing a workable strategy. When you consider all this, you might conclude that …

Myth #3: We shouldn’t do large-scale, unsupervised matching

Imperfect matching strategies, when applied automatically to real-world large datasets, might:

• Fail to discover some relationships (false negatives), an outcome that may not be terribly problematic. In the worst case scenario, we have wasted a great deal of effort developing matching strategies that do not improve our metadata.
• Create incorrect relationships between items (false positives), what seems like a potentially larger problem, where we have added incorrect relationships to the metadata.

Many have the instinct to avoid false positives at any cost, even if this means missing many additional correct relationships at the same time. They might come to the conclusion that if we cannot have 100% precision (see our previous myth), we simply should not allow matching strategies to act in an automated, unsupervised way on large datasets. While there might be circumstances where this belief is rational, in the context of the scholarly record, this notion is seriously flawed.

First, if you are dealing with any medium to large-sized dataset, it almost certainly contains errors, even before you apply any automated processing to it. Even if data is submitted and curated by users, they can still make mistakes, and might themselves be using automated tools for extracting the data from other sources, without your knowledge. It is thus not entirely obvious that applying an (imperfect) matching strategy to create more relationships would actually make the data quality worse.

Second, while we cannot eliminate all matching errors, we can place a high priority on precision when developing strategies, with the aim of keeping the number of incorrectly matched results as low as possible. We can also make use of additional mechanisms to easily correct for incorrectly matched results, for example doing so manually, in response to error reports.

Finally, the results of matching should always contain provenance information to distinguish them from those that have been manually curated. This way, the users can make their own decisions about whether to use and trust the matching results, relative to their use case.

By applying those additional checks, we can minimize the negative effects of incorrect matching, while at the same time reap the benefits of filling gaps in the scholarly record.

Myth #4: We can only ever guess at the accuracy of our matching results

In attempting to determine the correctness of our matching, we immediately encounter a number of inherent limitations. The sheer amount of entries in many datasets prevents a thorough, manual validation of the results, but if instead, we use too few or specific items as our benchmarks, these are unlikely to be representative of overall performance. The unpredictable nature of future data adds another wrinkle: will our matching always be as successful as when we first benchmarked it or will its performance degrade relative to some change in the data?

With so many unknowns, are we then doomed? No! We have rigorous and scientific tools at our disposal that can help us estimate how accurate our matching will be. How do we use them? Well, that is a big and fairly technical topic, so we will leave you with this little cliffhanger. See you in the next post!

Optica Publishing Group was one of the earliest publishers to send ROR IDs to Crossref in DOI metadata. In this interview, we speak with them to learn more about their rigorous processes for cleaning organization names.

Amanda French

Chris and Sasha, can you tell me your names, titles, and organizations?

Chris Iannicello

Chris Iannicello, Director of Business and Content Intelligence, Optica Publishing Group.

Sasha Schwarzman

My name is Sasha Schwarzman. My position is Content Technology Architect with the Optica Publishing Group.

Amanda French

Tell us about Optica Publishing Group. When was it founded, what does it do, and what makes it unique?

Sasha Schwarzman

Optica was founded in 1916 as the Optical Society of America, and the first issue of the Journal of the Optical Society of America was published in 1917. In 2021, the organization’s name was changed to Optica to reflect the society’s global membership, and Optica Publishing Group became its publishing arm.

Chris Iannicello

The unit we work in, Optica Publishing Group, is responsible for publishing optics and photonics scientific journals and books. There’s also an outreach program that’s affiliated with the society, which is a nonprofit, and that involves hosting meetings and conferences. The society also engages with young scholars to promote the field of optics and photonics. We also get involved with legislation in Congress related to issues that affect scientific research and funding.

Amanda French

Wonderful. And how many journals and publications does Optica Publishing Group produce?

Sasha Schwarzman

Optica Publishing Group’s portfolio offers the largest peer-reviewed optics and photonics content collection. It includes 21 peer-reviewed publications, co-publications, and papers from more than 1,150 meetings and conferences, including associated videos. The content comprises more than 460,000 journal articles, conference papers, and videos.

Amanda French

Gotcha. So when did you hear about ROR for the first time?

Chris Iannicello

A few years ago, I was attempting to standardize our affiliation data a little better – clean it. And I came across GRID at that time, maybe three or four years ago. I started using GRID IDs informally in my own cleaning exercises against our affiliation data. And by affiliation data, I mean both author institutions and also institutions that subscribe to our content. Both those lists require different degrees of cleaning. Then I heard the news that GRID was being sunset, and I started learning about ROR. So about two years ago, I really started to use ROR exclusively, once I knew GRID was going away.

Sasha Schwarzman

I think I heard about the ROR system four years ago, and I believe it was at a JATS-Con conference that I attended. At JATS-Con, we always talk about persistent identifiers. When ROR came along, of course, there was an announcement, and then there was, I believe, a presentation from the ROR folks. And ever since, it has been on my radar. Then, when Crossref announced that the Funder Registry is going to be sunset, that really caught my attention. Also, I’m on the JATS4R Author and Affiliations working group. As part of discussions leading to the release of Version 2, we talked a great deal about ROR being probably the dominant identifier for affiliations in the near future.

Amanda French

Ah, yes, I’m so glad you mentioned that, because we have looked at the JATS4R proposals about affiliations during the open comment period, and they looked good to us, so we’re pleased to know that ROR is such a key part of that. I think we didn’t leave any comments, because we were just pleased with them as they were.

Sasha Schwarzman

We did retain examples with ISNI and Ringgold, because you have to include those.

Amanda French

Yes. Understandable. Chris, you mentioned that you were using first GRID and then ROR in your cleaning projects for both author affiliations and institutional subscribers. I’m curious about what that cleaning involves and why it’s important.

Chris Iannicello

The second question is a lot shorter to answer than the first, so I’m going to give the answer to the second question first, which is, the messier the data, the more we need ROR. For years, when we accepted submissions for manuscripts, which may or may not become a published article, the author could submit what institution, college, university, or laboratory they’re affiliated with. It was a free-form field, and they could put as much text as they wanted. So we have data that may mention the university in it, but it also has the address, the PO box, the phone number, and it’s one piece of metadata, it’s not split up in our database. So the value of ROR to us is just to standardize the identity of all these places. We may have hundreds of different permutations of the same institution. To me, that’s the big value: to have a way of quality checking the information we have internally.

The information we’re asking for now is cleaner, because not only have we required cleaner data, but we’ve also integrated the ROR API so that authors can choose a ROR affiliation when they submit a manuscript, although not a large percentage is doing that as of yet.

The answer to the first question about the cleaning process is that it’s a very involved process where I take that metadata that I mentioned, which has a lot of information, and I attempt to extract the university or the laboratory or the the part of the metadata that is actually saying what the institution or affiliation is. And that is a multi-step cleaning process that has nothing to do with ROR: it just has to do with delimiting our data to the point where I can find the best possible part of the metadata of that might be an institution. Once I do that, I take that list and I put it against the ROR data. Sometimes I’ll get a one-to-one text match, and we consider that a match. If it’s not a one-to-one text match, then I run it against your API, line by line. And then I review those results manually.

Amanda French

Which is fantastic, by the way. Not everybody does that much work to get to that level of quality. I have to say, too, you’re by no means alone in having accepted affiliations as text strings for so long. I think that’s an endemic problem in publishing that for many years, even after the move to digital, people had a free-text field for that kind of data.

Chris Iannicello

When I get to the point of manual review, I have the cleaned affiliation string from our data, and the first thing I’ll do is see if there’s a URL involved, because that’s a much cleaner way to find a match, since you don’t have to worry about permutations of names and the URL changes a lot less often than the name does. If that doesn’t work, then I’ll just start Googling the name until I find something that looks like it has a ROR ID.

In that process, I’m searching the harvested ROR data that I have taken from your data dump, which I put into a business intelligence tool called Tableau, and I’ve included the hierarchy data, which helps. I try to run it up to the highest parent in the hierarchy chain when I assign the ROR ID or name. Sometimes I’m not able to do that, just because it’s too time-consuming and might not be granular enough anyway (for example, the “US Department of Defense” might be too broad as opposed to its children like “US Army” and “US Navy”).

I have a mechanism where you can see the child or the parent: you can toggle between the two, so if you want to see lower numbers, more bundled, you toggle to the parent for things like article counts and author accounts. The granularity might not be perfect, because I only have two data points, the institution I found a match for and its respective parent, no matter how many generations it is up the chain to that parent. I don’t specify that.

Once I do all that, then I can start reporting on the data and presenting it to stakeholders internally. I give them the choice between looking at the ROR ID or looking at the raw strings in the same row broken out if they ask for it, which obviously makes it a much longer report. Basically, those are the major steps.

Amanda French

I see.

Chris Iannicello

I don’t do any automated matching, because the information we’re using this for is too important. We’re not doing broad trending with this data. We’re doing reports that require very specific article counts, author affiliations, and articles per author affiliation per year. We need exact numbers or as close as we can get to it. I’ve been on the webinars where you’ve discussed automatic matching, and I’ve heard all the techniques people are using, and they’re great, and I understand why they’re doing well. But we just have to trust the data 100% for our use cases.

Amanda French

What are those use cases? In other words, what’s the purpose of your reports?

Chris Iannicello

We’re using them for prospective business arrangements. Sometimes it’s with a single university, sometimes it’s with a consortium, sometimes it’s with a whole country for Read-and-Publish deals or Publish-only deals. In those deals, when you’re discussing the details, it’s very important to know the historical counts of authors who have written articles through Optica Publishing Group, because those counts determine the basis for the amount of the deal.

If you’re not far along in the negotiation process, you may just want an order of magnitude, so you can do a quick keyword search of affiliations that may have more than one name. Now that might be done against ROR data, or that might be done against our clean affiliation data, or it might be both. That’s more informal. But when you get closer to finalizing a deal, then the exact counts become important.

But we also want to apply ROR against our internal customer data, against all of the Optica society members, all of whom have affiliation data. If we take a cross-section of all our members, what are the ten leading affiliations that they’re from? I’ve run the ROR API against those lists, and I’ve taken fuzzy matches on that, and in the reports I’ve specified that these are fuzzy. Maybe one-tenth of them are clean. I’ve set up the reports so you can toggle between the fuzzy matches and the clean matches, so you can remove the fuzzy matches from the report completely, and then you’re only looking at the ones that I manually checked. It’s a toggle that says, “manually checked” or “fuzzy match.” And I uncheck the fuzzy match by default, because I know it’s risky to report that. So, if they just want to see general trends, they can include fuzzy matches.

A related use case for that data is to try to identify organizations that have a lot of authors who publish articles with us but do not subscribe to our content.

Amanda French

That’s a great use case, of course, because it’s not even just a business opportunity, like “Hey, maybe you should subscribe.” That’s good for research, because if those researchers are publishing in your journals, then they probably want to be reading them as well.

Chris Iannicello

Yes, that’s the whole sales pitch. Researchers from your institution have published thirty articles in the past two years in our journals, but if they go to read them in your libraries, they can’t get to them, or at least not to the non-open-access ones, anyway.

Amanda French

Right.

Sasha Schwarzman

I want to elaborate on one of the use cases that Chris mentioned, which is the case where we already use ROR IDs for funding purposes. Specifically, as far as open access is concerned, there are two business models. Either it’s a funder who pays for open access, or it is an institution that pays for open access. When the institution pays, there are two variations. The first is a Read-and-Publish deal with a library consortium or with a particular institution that pays for open access. The second variation is a Publish-only deal, which can apply to open-access and hybrid journals.

Because, in both cases, it is an institution rather than a funder that pays for open access, it is crucial to identify the institution correctly, and we use ROR IDs to do so. We have already implemented that, and thanks to Chris’s efforts, we have very strict one-to-one correspondence between an institution and a ROR ID. If the ROR ID belongs to a particular Read-and-Publish deal, we send the information to RightsLink, a Copyright Clearance Center service. They clear it with the institution, and then the publisher gets paid.

As part of the extensive quality control that each journal article undergoes, I use Schematron to check the correspondence between an eligible affiliation and its ROR ID. If they don’t match, there is a QC option to reconcile them based on the ROR Registry. That is an actual use case where we already use ROR IDs for those Read-and-Publish or Publish-only deals.

Of course, as Chris said, we can use this data to know which institutions have published with us but haven’t signed a deal with us yet. Or maybe the deal needs to be renegotiated because we’re paid for this number of authors, but many more have published with us, so maybe we need to renegotiate the terms. That is very important from the business point of view.

Amanda French

Yes. I’m curious about RightsLink. Do they accept ROR IDs as an input, or do they only accept names?

Chris Iannicello

Both. We can specify three different identifier types to verify if an accepted manuscript is a match on the RightsLink side. It’s the name, the ROR ID, or the email domain. Or more than one if needed.

Amanda French

Interesting!

Chris Iannicello

We’re giving them the ROR ID and the ROR name. If there’s a submitted manuscript that comes in from a region that we may have a deal with, where the author didn’t choose a ROR value when they submitted, then I get an alert. And I’ll go and manually match it, which I can do in our internal production system. It happens pretty often.

Now, that doesn’t mean they’re part of the deal, but they’re from an area where they might be. I have configured the system to get alerts for certain areas, and, worst-case scenario, they are not part of the deal, and now we just have one more matched ROR in our data. And if I had more time, I’d go to every single paper that wasn’t matched and do that, but that’s not on my radar right now. We certainly could do that, because the ROR API is very helpful in that regard with the autofill and the drop down, and now we have the related organizations, so it’s really a very useful tool just for matching purposes.

Amanda French

That’s great. The other thing that I’m really curious about, and I don’t know if you know this, but Optica Publishing Group is one of the top contributors of ROR IDs in DOI metadata to Crossref. We haven’t yet talked about the fact that you’re doing that, that you’re making sure to include author affiliations with ROR IDs in Crossref metadata. Can you tell me a little bit more about why and how you do that?

Sasha Schwarzman

As far as “why” is concerned, it’s because of the culture of the OPG in general, and my responsibility in particular, is to ensure that all the metadata has integrity and quality. We want to be good citizens of our publishing community and contribute as much metadata as possible, as much correct metadata as possible, to Crossref as an exchange mechanism among publishers. We try to be as accurate as possible, and we don’t see why we wouldn’t contribute this data, since it goes through the rigorous quality control that Chris has described and through other processes to make sure our persistent identifiers are correct.

As far as “how,” we have a process called fulfillment. Our colleague Jennifer Mayfield ensures that all metadata that goes to Crossref is correct. If there are any problems, Crossref reports them, and then we redeposit our metadata so that everything is accurate.

Amanda French

And I would imagine that you are probably doing XML registration with Crossref?

Sasha Schwarzman

Yes, definitely. We use a subset of JATS and then transform it to the XML that Crossref accepts.

Amanda French

What’s next for you with ROR? I think you’re working on cleaning up your funder data and matching that to ROR IDs?

Sasha Schwarzman

I’ll start by explaining what we currently do, which will help clarify what we need to do moving forward and how ROR IDs will factor into that. We currently use the Open Funder Registry to assign DOIs to funders. We go through the registry from top to bottom, categorizing funders into three main groups: those requiring a CC BY license, those participating in CHORUS, and the frequent funders of optics and photonics research. Our goal is to ensure we’re capturing all relevant funders.

We maintain a list of top ancestors in each category, and this list evolves over time. For instance, we might learn that a funder now requires a CC BY license or there is a new CHORUS participant. The license information varies depending on these three categories. Each funder has descendants, sometimes quite a few. If an ancestor requires a CC BY license, then all its descendants do, too. So, each time a new release of the Open Funder Registry comes out, we go through the registry to find all the descendants in each category. Some funders are particularly dynamic. For example, the United States Department of Defense has frequent changes, and in the United Kingdom, certain funders in the CC BY category also change often.

We use Schematron technology, where we say, “All right, we found this funder.” We’ve integrated the Open Funder Registry widget into our submission system, so the XML data usually includes the funder DOI. However, sometimes, an author can’t find a match for a funder and types something in that’s not quite correct. Or, during production, an author may realize they forgot a funder and then provide its name but not the DOI. In these cases, using Schematron, we ensure that the funder’s name matches its DOI. If there’s no match, we prompt the user with the most likely DOI, using “preferred label,” that is, the official name of the funder. This step is critical because it’s important to have the correct license, like a CC BY license, or to know if it’s a CHORUS funder, which may affect the embargo period and so on.

Now, with the news that the Open Funder Registry will sunset and be replaced by the ROR Registry, my focus in the coming months will be figuring out how to replicate or redesign this process to maintain the quality control we need. The first question is about coverage. Thanks to Chris, we have many years’ worth of data on all the funders of research published in our journals. Are they all in the ROR Registry? We’ve been working with you to identify any missing ones because we can’t switch until the ROR Registry covers a significant portion of what we have.

We also need to figure out how to manage this process. How do we find those descendants? What tools do we need? Should we use the ROR API or the data dumps, which I imagine will be similar to the Open Funder Registry releases? These are very different data sources. In the coming months, I’ll be exploring the ROR Registry to determine whether to use the API or the data dumps and how to recreate or maybe even invent new processes that give us the same level of control we currently have with the Funder Registry.

Amanda French

Oh, that’s exactly what I wanted to hear. We’re hoping to work quite closely with you to make sure that all of the funders that you need to have in ROR are in ROR, assuming they’re in scope. We’ve had some correspondence with Chris about this already.

The last I checked, about 54% of Funder Registry records have equivalents in ROR, and those are the most frequently used funders. We’re continuing to go down the list of less frequently used funders and updating ROR to include them when possible.

We’ve been talking with CHORUS, as well, and they are also in the process of building ROR into their workflows. They’ve got it in some of their services and in their public API. And on the recent Crossref community call, Crossref emphasized that they’re prioritizing accepting ROR IDs in the schema where they currently accept Funder IDs.

Sasha Schwarzman

By the way, this is in Chris’s purview, but we have years and years and years of data about funding based on the DOIs of those funders, and then when we switch, there will be years and years of data based on ROR. And I wonder if we need to have business intelligence covering the range of years whether we will have to account for two identifiers for the same funder, which may or may not be a challenge.

Chris Iannicello

I’m not sure. I mean, it’s pretty straightforward to look for one and then the other if the first one was missing. But as far as my part in this effort, we have about 13,000 different funder names, and maybe half of them have a DOI. And we’re focusing on the ones that have a DOI for now, and of those 6000 or so that have a DOI, I’ve matched about 4500 of them. Most of them were not matched manually: I was able to find a match through the DOI in ROR through an automatic name match. I was only able to manually match a couple hundred of those so far, but I haven’t spent a lot of time on that part of it yet. The remaining 1400 or so I will probably match manually through the year. And the ones that don’t have a DOI we may never match, as it’s just very time-consuming to match it that way. Maybe we’d apply a fuzzy match to it, if someone requests some kind of high-level reporting.

Amanda French

Also, we are working on an additional service to match names to ROR IDs. We’re doing that in conjunction with Crossref. Our API, as you’ve mentioned, Chris, does do a good job of matching, but we’re looking at developing a matching service that will do an even better job of that.

One question I’d love to ask, and we’ve touched on this already: What do you hope ROR does in the future? What problems could we fix for you, or what changes could we make that would make your lives easier?

Chris Iannicello

Any type of improvement for matching would be great. Perhaps a mechanism to take in verified matches from me and others to use as training data for some kind of machine learning tool. We’ve dabbled in that a little bit, internally, and the results were not good, so it may be a non-starter. I’m assuming that it won’t be a non-starter forever; if the AI is not up for the challenge now, I’m sure it will be soon enough. That might be something to pursue down the line, because I can provide about 5,000 manually matched records. There might be a mistake in there, but I’ve looked at each one. It could be a basis for all kinds of matching down the line, perhaps.

Amanda French

No, I think you’re really right. We have done some of that with the American Physical Society. They also did a lot of heavy curation where you can be really sure that this particular variant of this name matches this ROR ID. We have a GitHub repository where we store both the code and the data for that matching service that I mentioned, and in the test_data folder, there are some datasets. The APS dataset has some verified matches that they have manually curated in the same way that you do. The Crossref publisher assertions are from the Crossref API where the publisher has asserted that this name matches this ROR ID. We think that’s probably less reliable than one that was manually reviewed by the publisher. I’ll have to see if we could use some additional datasets for that project or if it’s too far along.

Chris Iannicello

Sounds good.

Sasha Schwarzman

One particular challenge is dealing with Chinese funders. Funders there change rather frequently, and because of translation challenges it’s not always clear whether the author specified the official funder’s name and what funder that is. That is, I think, a particular challenge in dealing with Chinese funders.

Amanda French

Yes, that’s been a priority for us. As you know, in our March community call we had a presentation from Jackson Huang, who was a LEADING Fellow with us and is now a member of our Curation Board, and who is a Chinese speaker who’s done some work on that. The focus wasn’t specifically on funders; but more like Chinese-language organizations in ROR generally.

What else would you like to say about ROR, or about the Optica Publishing Group, or about metadata, or about anything at all?

Chris Iannicello

I don’t have anything else other than it’s been a really great tool, and it’s really helped our efforts, so we really appreciate that it exists.

Sasha Schwarzman

I’m looking forward to unifying the IDs for the funders and affiliations. It does make sense. I realize it will be challenging, but in the long run, I think it’s the right thing to do, especially because the ROR Registry is actively and openly curated and has an active user base. So I am looking forward to this development and to this challenge.

Amanda French

Wonderful. Yes. I’m glad to hear that. I think it’s going to be very exciting to see everything that comes out of this. Thank you both for taking the time to talk to me.

Chris Iannicello

And we thank you so much.

Sasha Schwarzman

Thank you very much!

Several widely used scholarly systems are using ROR IDs to identify and disambiguate funding organizations. In this post, we’ll give you a glimpse of how and why these systems use ROR IDs as funder identifiers.

You might already know that for the last ten years, the Crossref Open Funder Registry (formerly FundRef) has been the default identifier for funding organizations. You might also know that last fall, Crossref announced “a long-term plan to deprecate the Open Funder Registry” in favor of ROR so that publishing systems and workflows can use one open registry, not two, to identify organizations.

If you’re interested in learning more about that transition, you can consult our guide to the Transition from Open Funder Registry to ROR or read previous ROR blog posts. But if you’re interested in learning more about early adopters who are already using ROR to identify funders now, read on.

Repositories are ahead of the curve

In general, repository systems are doing better than publishing systems in terms of adopting ROR for identifying funding organizations. One major factor in this phenomenon is undoubtedly DataCite’s early support for this use case. DataCite, the DOI registration agency that most data repositories use, has supported ROR identifiers in the fundingReferences element of its metadata schema since August of 2019. This early schema change encouraged data repositories to send ROR IDs to DataCite in DOI metadata. Note, too, that DataCite is now encouraging funders to register DOIs for awards, and ROR is DataCite’s recommended funder identifier in award metadata.

DataCite’s API shows that nearly one-third of DataCite DOIs with an identifier in the funding reference use ROR as the funder identifier. Out of a total of about 1.4 million DataCite DOIs with funding identifiers, over 432,000 of them include ROR IDs.

Another factor that has driven early adoption of ROR for funder identification in repository systems is the U.S. National Institutes of Health (NIH) Generalist Repository Ecosystem Initiative (GREI), whose primary aim is to develop “a common set of cohesive and consistent capabilities, services, metrics, and social infrastructure across generalist repositories” so that NIH-funded researchers can more easily discover and share their research data. Seven generalist repository systems are participating: Dataverse, Dryad, Figshare, Mendeley Data, the Open Science Framework (OSF), Vivli, and Zenodo. All seven have agreed to implement ROR for author affiliations, and some have also taken the opportunity to implement ROR for funder identification sooner rather than later. Dryad, Figshare, Zenodo, and Vivli implemented ROR to identify funders in 2023; Dataverse and OSF are actively planning to use ROR for funders soon, and Mendeley Data will do so at some point in the future. When data repositories use ROR as a standard funder identifier, it enables all funders, including the NIH, to track research datasets resulting from their funding more easily.

DataCite’s API shows that all but one of the top five providers of ROR IDs for funders are GREI participants: Dryad, Zenodo, and Figshare together have registered over 100,000 DOIs that use ROR IDs in funding references. An additional repository operated by the Leibniz Institute DSMZ, StrainInfo (a collection of data about microorganisms and cell cultures), has used ROR IDs for funder identification in over 300,000 DataCite DOIs.

Let’s take a look at each of these systems to see how they’ve implemented ROR for funders.

Dryad is the first to switch to ROR for funder IDs

As the very first system to adopt ROR in 2019, Dryad has been using ROR for author affiliations longer than anyone else, and Dryad was almost as quick to adopt ROR for funding identification. Like many publishing and repository systems, Dryad originally used Crossref’s Open Funder Registry to identify funders, but with the announcement that the Open Funder Registry is headed for deprecation, Dryad moved speedily to switch from Funder IDs to ROR IDs, completing the transition in October 2023. This transition involved four main tasks:

• Changing Dryad’s submission form funder lookup to use ROR instead of the Funder Registry
• Converting existing Funder IDs in Dryad’s data to ROR IDs
• Enabling the capacity to send ROR IDs in new metadata deposits to DataCite
• Updating the funder identifiers in Dryad’s existing DataCite DOI records

We can see the result of Dryad’s work in their submission form. Now, in Dryad, the typeahead fields for “Institutional affiliation” and “Granting organization” both use ROR, and both helpfully reveal organization names, acronyms, and locations based on what the user types, just as we recommend in our form guidelines.

Dryad is the only system of those we’re discussing here to have transitioned entirely from using the Crossref Open Funder Registry for funder identification to using ROR for funder identification: other systems have either started fresh with ROR or continue to use multiple registries.

Zenodo’s funder model is based on ROR

Also in October 2023, the generalist data repository Zenodo completed its migration to an open source research data management system called InvenioRDM. InvenioRDM was architected from the beginning with ROR for both author affiliations and funders, and other repositories that use InvenioRDM (e.g., CaltechData) therefore have similar capabilities as Zenodo.

Zenodo takes an “award-first” approach to connecting items to funders, asking the user to search for and select a specific award and then using the award identifier to pull in information about the funder who granted that award. Users also have the option to filter awards by a limited number of funders who participate in OpenAIRE. Zenodo’s “standard award” field pulls award and funder data from the OpenAIRE Graph dataset, which uses ROR as one of several organization identifier schemes.

Users can also add a custom award – for instance, one that isn’t provided by OpenAIRE – and the custom award lookup in Zenodo is powered exclusively by ROR data to suggest funders in the award lookup funder field. Zenodo shows both the country code and the unique portion of the ROR identifier next to the name of the funder in the list of suggestions.

Zenodo communities can also be hosted by funders, and these funders can also be identified by ROR IDs, as is the case for the GREI community managed by NIH.

Zenodo also exposes funder information in its API at https://zenodo.org/api/funders.

It’s important to reiterate that this functionality isn’t confined to Zenodo: the ROR-powered funder lookup is built into the system that runs Zenodo, InvenioRDM, so that anyone who uses InvenioRDM can also use ROR to identify funders. InvenioRDM’s excellent documentation explains in detail how award and funder lookups work in the system and gives more detail about the ROR-based data model for funders.

InvenioRDM is the first new repository system to launch with ROR as the basis of its organizational data model, but it surely won’t be the last new repository system to do so.

Figshare imports ROR from Dimensions

Both Figshare’s central repository at figshare.com and Oxford University’s Figshare-powered institutional repository at https://portal.sds.ox.ac.uk/ (also available via https://oxford.figshare.com) are contributing a significant number of ROR IDs as funder identifiers to DataCite metadata. Figshare, like Zenodo, asks the user to enter an award number or title and then pulls in funding information based on that award; in this case, Figshare pulls the grant title, grant ID, funder name, and ROR ID from Dimensions grant data and includes it in the Figshare item’s metadata record.

ROR was originally seeded with the GRID organization identifiers and data used by Dimensions, so there is a close relationship between ROR IDs and GRID IDs, and most ROR and GRID records are mapped to each other. Dimensions’s support documentation explains how ROR IDs are incorporated into its data:

The curation work that Dimensions performs means that organization information in Figshare, including ROR metadata for funding organizations, remains up to date.

StrainInfo uses ROR for funders

The largest provider of ROR IDs in DataCite funding references is a repository called StrainInfo, which is “a service developed to provide a resolution of microbial strain identifiers by storing culture collection numbers, their relations, and culture-associated data.” StrainInfo is currently supported by the NFDI4Microbiota consortium, which is funded by the German Research Foundation (DFG), and is developed and maintained by Leibniz Institute DSMZ.

StrainInfo indexes more than 290,000 microbial strains, each of which has its own DOI registered with DataCite, and each of which contains a funding reference in the metadata to award 460129525 issued by the German Research Foundation (DFG), whose ROR ID is https://ror.org/018mejw64.

Thus, while StrainInfo is currently responsible for the largest number of DOI records with ROR IDs in the funderIdentifier field to DataCite, all 300,000+ of those records (including versions with updated data for several strains) reference a single funder with a single ROR ID. As Ted Habermann noted in his 2019 piece How Many RORs Do We Need?, it’s good news for ROR adoption that “some DataCite members only need to know a small number of RORs,” since this can make implementation of ROR quite a bit easier.

Many more systems use ROR for funders

The systems reviewed here are by no means the only ones using ROR IDs to identify funders. Here’s an alphabetical list of additional systems that are currently using ROR IDs for funder identification:

• CHORUS Dashboard and Reporting Services
• ChronosHub Open Access Management Platform
• DOE Award DOI Service
• DSpace 8
• DSpace-CRIS
• FAIRsharing
• Genomes OnLine Database (GOLD)
• Japan Link Center (JALC)
• MasterVision and PaperStack by DataSalon
• OA.Report
• OpenAlex
• ORCID
• ProposalCentral by Altum
• RADAR
• Sherpa Services
• Vivli Center for Global Clinical Research Data

See the full list of ROR Integrations to see more systems that are using or planning to use ROR for all kinds of purposes beyond funder identification – author affiliations, Open Access deal management, publisher identification, internal statistics, and more. If you’re managing a system that uses ROR for funder identification and you don’t see it listed here, let us know!

What’s next

More and more systems will be using ROR to identify funders in the coming months and years as the deprecation of the Open Funder Registry nears. Crossref has been working hard on building support for publishers to deposit ROR IDs as funder identifiers, and this capacity will be released before the end of the year. Moreover, as more and more funders register DOIs for awards using DataCite or Crossref, they will undoubtedly begin by using ROR IDs to identify their own organization as well as other funding institutions.

“Open funding metadata is arguably the next big thing,” avers a recap of a funding metadata workshop convened by Crossref last year, for many reasons: “meta-researchers need this information in order to analyze research on research, editors are concerned with research integrity, including funding trends, and funders themselves need to track the reach and return of their support.” This year alone, several lively discussions of how to improve funding metadata through the use of persistent identifiers in published research have been held, including:

• ROR Annual Meeting, Why We All Need Good Funding Metadata, January 2024
• Altum webinar, Advancing Research with Persistent Identifier (PID) Innovations, February 2024
• CHORUS Forum, Navigating the Future of PIDs in Scholarly Publishing, May 2024

The other commonly mentioned “next big thing,” AI, or more precisely machine learning, also has a bearing on funding metadata: Crossref and ROR are collaborating on an exciting project to build and evaluate robust, large-scale matching strategies that can reliably connect tens of thousands of funder names in Crossref metadata to ROR IDs. Nevertheless, because any such automated post-processing of metadata is bound to have some errors, it’s important that good funding metadata be created and reviewed by humans as far upstream as possible, ideally during the manuscript submission process or publishing production process.

So, in short, if you’re in charge of a scholarly publishing system or workflow that’s using the Open Funder Registry or one that isn’t using funder identifiers at all, you should be thinking about using ROR. We’re here to help!

• Are you using the Open Funder Registry? Start by consulting our Funder Registry Transition Guide.
• Are you new to persistent identifiers and/or ROR? Start by browsing through our technical documentation.

ROR is a responsive, open, community-led endeavor, and we’re happy to work with you to integrate ROR IDs into your scholarly publication metadata.

We’ve just released an update to our participation report, which provides a view for our members into how they are each working towards best practices in open metadata. Prompted by some of the signatories and organizers of the Barcelona Declaration, which Crossref supports, and with the help of our friends at CWTS Leiden, we have fast-tracked the work to include an updated set of metadata best practices in participation reports for our members. The reports now give a more complete picture of each member’s activity.

What do we mean by ‘participation’?

Crossref runs open infrastructure to link research objects, entities, and actions, creating a lasting and reusable scholarly record. As a not-for-profit with over 20,000 members in 160 countries, we drive metadata exchange and support nearly 2 billion monthly API queries, facilitating global research communication.

To make this system work, members strive to provide as much metadata as possible through Crossref to ensure it is openly distributed throughout the scholarly ecosystem at scale rather than bilaterally, thereby realizing the collective benefit of membership. Together, our membership provides and uses a rich nexus of information — known as the research nexus — on which the community can build tools to help progress knowledge.

Each member commits to certain terms, such as keeping metadata current, updating links for their DOIs to redirect to, linking references and other objects, and preserving their content in perpetuity. Beyond this, we also encourage members to register as much rich metadata as is relevant and possible.

Creating and providing richer metadata is a key part of participation in Crossref; we’ve long encouraged a more complete scholarly record, such as through Metadata 20/20, and through supporting or leading initiatives for specific metadata, like open citations (I4OC), open abstracts (I4OA), open contributors (ORCID), and open affiliations (ROR).

Which metadata elements are considered best practices?

Alongside basic bibliographic metadata such as title, authors, and publication date(s), we encourage members to register metadata in the following fields:

References

A list of all the references used by a work. This is particularly relevant for journal articles but the references can include any type of object, including datasets, versions, preprints, and more. Additionally, we encourage these to be added into relationships, where relevant.

Abstracts

A description of the work. These are particularly useful for discovery systems that will promote the work, and are often used in downstream analyses such as for detecting integrity issues.

Contributor IDs (ORCID)

All authors should be included in a work’s metadata, ideally alongside their verified ORCID identifier.

Affiliations / Affiliation IDs (ROR)

Members are able to register contributor affiliations as free text, but we are encouraging everyone to add ROR IDs for affiliations as the recommended best practice, as this differentiates and avoids mistyping. These two fields have newly been added to the participation reports interface in the most recent update.

Funder IDs (OFR)

Acknowledging the organization(s) that funded the work. We encourage the inclusion of Open Funder Registry identifiers to make the funding metadata more usable. This will evolve into an additional use case for ROR over time.

Funding award numbers / Grant IDs (Crossref)

A number or identifier assigned by the funding organization to identify the specific award of funding or other support such as use of equipment or facilities, prizes, tuition, etc. The Crossref Grant Linking System includes a unique persistent link that can be connected with outputs, activities, people, and organizations.

Crossmark

The Crossmark service gives readers quick and easy access to the current status of a record, including any corrections, retractions, or updates, via a button embedded on PDFs or a web article. Openly adding corrections, retractions, and errata is critical part of publishing, and the button provides readers with an easy in-context alert.

Similarity Check URLs

The Similarity Check service helps editors to identify text-based plagiarism through our collective agreement for the membership to access to Turnitin’s powerful text comparison tool, iThenticate. Specific fill-text links are required to participate in this service.

License URLs

URLs pointing to a license that explains the terms and conditions under which readers can access content. These links are crucial to denote intended downstream use.

Text mining URLs

Full-text URLs that help researchers in meta-science easily locate your content for text and data mining.

What is a participation report?

Participation reports are a visualization of the data representing members’ participation to the scholarly record which is available via our open REST API. There’s a separate participation report for each member, and each report shows what percentage of that member’s metadata records include 11 key metadata elements. These key elements add context and richness, and help to open up members work to easier discovery and wider and more varied use. As a member, you can use participation reports to see for yourself where the gaps in your organization’s metadata are, and perhaps compare your performance to others. Participation reports are free and open to everyone - so you can also check the report for any other members you are interested in.

We first introduced participation reports in 2018. At the time, Anna Tolwinska and Kirsty Meddings wrote:

Metadata is at the heart of all our services. With a growing range of members participating in our community—often compiling or depositing metadata on behalf of each other—the need to educate and express obligations and best practice has increased. In addition, we’ve seen more and more researchers and tools making use of our APIs to harvest, analyze and re-purpose the metadata our members register, so we’ve been very aware of the need to be more explicit about what this metadata enables, why, how, and for whom.

All of that still rings true today. But as the research nexus continues to evolve, so should the tools that intend to reflect it. For example, in 2022, we removed the Open references field from participation reports after a board vote to change our policy and update the membership terms meant that all references deposited with Crossref would be open by default. And now we’ve expanded the list of fields again, adding coverage data for contributor affiliation text and ROR identifiers.

Putting it in practice

To find out how you measure up when it comes to participation, type the name of your member organization into the search box. You may be surprised by what you find—we often speak to members who thought they were registering a certain type of metadata for all their records, only to learn from their participation report that something is getting lost along the way.

You can only address gaps in your metadata if you know that they exist.

More information, as well as a breakdown of the now 11 key metadata elements listed in every participation report and tips on improving your scores, is available in our documentation.

And if you have any questions or feedback, come talk to us on the community forum or request a metadata Health Check by emailing the community team.

American Physical Society Becomes Largest Society Publisher to Adopt Research Organization Registry Identifiers

The American Physical Society has adopted the Research Organization Registry identification system, assigning persistent identifiers to authors’ affiliations for every paper published in the Physical Review journals. The action comes as a response to challenges faced by the scientific community regarding metadata tracking.

Researchers, institutions, and libraries alike rely on high quality, accurate metadata to monitor publication output and impact. However, institutional metadata has lacked standardization across the publishing landscape. By assigning openly available ROR persistent identifiers, APS provides a solution to this problem and more. These identifiers will help authors and institutions keep track of the scope of their work.

APS also sends ROR identifiers to Crossref, an online record for scholarly metadata. The initiative makes relevant metadata discoverable across all search tools, with implications for funding and for participating in open access models.

APS is one of the first publishers to incorporate ROR identifiers in published metadata, demonstrating the society’s commitment to advancing open science.

Watch APS discuss their adoption of ROR

Last week at the ROR Community Call, APS Lead Data Analyst Arthur Smith explained why and how APS adopted ROR IDs for author affiliation identification.

The American Physical Society may be the largest society publisher so far to adopt ROR, but we know they won’t be the last. Kudos to APS for helping to lead the way.

Now that we’ve released v2 of the ROR schema, we have the power to add external identifiers to ROR records in the external_ids field without making a major schema change – and since we have the power, we want to use it! We’re therefore issuing a request for feedback on the requirements we should set for including additional external organizational identifiers in ROR records.

The four external identification schemes that are currently included in ROR are all global identifiers – they are unique identifiers that are used around the world. We have, however, received several requests to include regional and national identifiers in ROR records as well as other globally unique identifiers; one important aspect of the draft external identifier proposal is to establish the criteria that such “non-global” organizational identifiers should meet.

Note that we are asking chiefly for feedback on the characteristics that new organizational identifiers should have in order to be added to ROR, although we are also more than happy to accept suggestions of organizational identifier schemes to add to ROR. To suggest that ROR add a particular new organizational identifier that would be useful for you, submit an issue to the ROR roadmap and choose the issue template “Add External ID.”

To give feedback, please visit the proposal document and suggest changes to the text or add comments in the margin by August 16th, and do feel free to share the document with anyone you think might be interested. We’ll give a summary of the comments and a plan for moving ahead at the ROR Community Call in September.

We rely on the ROR community to keep us headed in the right direction – thanks in advance for contributing your time and expertise!

In our previous blog post about metadata matching, we discussed what it is and why we need it (tl;dr: to discover more relationships within the scholarly record). Here, we will describe some basic matching-related terminology and the components of a matching process. We will also pose some typical product questions to consider when developing or integrating matching solutions.

Basic terminology

Metadata matching is a high-level concept, with many different problems falling into this category. Indeed, no matter how much we like to focus on the similarities between different forms of matching, matching affiliation strings to ROR IDs or matching preprints to journal papers are still different in several important ways. At Crossref and ROR, we call these problems matching tasks.

Simply put, a matching task defines the kind or nature of the matching. Examples of matching tasks are bibliographic reference matching, affiliation matching, grant matching, or preprint matching.

Every matching task has an input, which is all the data that is needed to perform the matching. Input data can come in many shapes and forms, depending on the matching task. For example, all of the following could be inputs to a matching task:

Department of Molecular Medicine, Sapporo Medical University, Sapporo 060-8556, Japan

<fr:program xmlns:fr="http://www.crossref.org/fundref.xsd" name="fundref">
  <fr:assertion name="fundgroup">
    <fr:assertion name="funder_name">
      European Union's Horizon 2020 Research and Innovation Program through Marie Sklodowska Curie
      <fr:assertion name="funder_identifier">http://dx.doi.org/10.13039/501100000780</fr:assertion>
    </fr:assertion>
    <fr:assertion name="award_number">721624</fr:assertion>
  </fr:assertion>
</fr:program>

Everitt, W. N., & Kalf, H. (2007). The Bessel differential equation and the Hankel transform. Journal of Computational and Applied Mathematics, 208(1), 3–19.

{
  "title": "Functional single-cell genomics of human cytomegalovirus infection",
  "issued": "2021-10-25",
  "author": [
    {"given": "Marco Y.", "family": "Hein"},
    {"given": "Jonathan S.", "family": "Weissman", "ORCID": "http://orcid.org/0000-0003-2445-670X"}
  ]
}

Every matching task also has an output. For our purposes, this is almost exclusively zero or more matched identifiers. In the context of a specific matching task, output identifiers may be of a specific type (e.g. we might match to a ROR ID, and never to an ORCID ID). In some cases, there can be a certain target set as well (i.e. matching only to DataCite DOIs). The output identifiers can have different cardinality depending on the task, meaning that the matching task might allow for zero, one, or more identifiers as a result of matching to a single input.

A matching strategy defines how the matching is done. Multiple strategies can exist for a specific matching task. Compound strategies can run other strategies and combine their outcomes into a single result.

In some cases, we may also want the matching strategy to output a confidence score for each matched identifier. A confidence score represents the degree of certainty or likelihood that the matched identifier is correct, typically expressed as a value between 0 and 1. This score may help with post-processing or further interpretation of the results.

To summarise, the anatomy of the matching task can be diagrammed as follows:

How to specify a matching task

Whenever we plan the development or integration of a matching solution, it is good to begin by answering a few basic questions:

1. What problem do we plan to solve with our matching task? What would we call our matching task and how would we describe it?

2. What do we expect as the input for this matching task? Which input formats do we need to be able to accept? What information do we expect to find in this input?

3. What kind of identifiers should be output? Is there a target set of identifiers? Can our matching output zero/one/or multiple identifiers, and under what conditions might that occur?

These sound fairly simple, but the answers to these questions can be remarkably complex. Once one tries to apply these concepts to real-world problems, they might encounter several non-obvious challenges.

For example, one common concern is at what level we should define each matching task. Consider the following problems:

Matching bibliographic reference strings to DOIs.

Example input:

Everitt, W. N., & Kalf, H. (2007). The Bessel differential equation and the Hankel transform. Journal of Computational and Applied Mathematics, 208(1), 3–19.

Matching structured bibliographic reference to DOIs.

Example input:

{
  volume: "208",
  author: "Everitt",
  journal-title: "J. Comput. Appl. Math.",
  article-title: "The Bessel differential equation and the Hankel transform",
  first-page: "3",
  year: "2007",
  issue: "1"
}

Are those discrete matching tasks (unstructured reference matching vs. structured reference matching), or are they the same task (reference matching) that can accept different types of inputs (unstructured or structured)?

Similarly, let’s compare the following tasks:

Matching affiliation strings to ROR IDs.

Example input:

Department of Molecular Medicine, Sapporo Medical University, Sapporo 060-8556, Japan

Matching funder names to ROR IDs.

Example input:

Alexander von Humboldt Foundation

Are these different matching tasks (affiliation matching vs. funder matching), or the same task with different inputs (organisation matching)?

Defining the boundaries of a matching task can also be difficult. Consider, for example, the need to obtain ROR IDs for organisations mentioned in the acknowledgements section of a full-text academic paper. To begin, one may first extract the acknowledgement section from the full text, then run something like a named entity recognition (NER) tool to isolate the organisation names from the extracted text, and finally match these names to ROR IDs. Is this entire process matching, with the input being the full text of a paper? Or perhaps matching starts with the acknowledgement section as the input? Instead, is it only the last phase, where we try to match the extracted name to the ROR ID, that constitutes the matching task, with the extraction phases being completely separate processes?

There are also important questions related to the expected behaviour of a matching strategy. Consider, for example, developing an affiliation matching strategy where we define our input as “an affiliation string”. What should happen when the strategy gets something else on the input, for example, song lyrics? Perhaps the strategy should simply return no matches, or an error, or we could say that in such a situation the behaviour is undefined and it simply doesn’t matter what is returned. But what should happen if in this input we have the lyrics of Street Life by Roxy Music, a song that mentions the names of a few universities that happen to have ROR IDs?

It is likewise important to consider what should happen if different parts of the input match to different identifiers, like in the following example:

Department of Haematology, Eastern Health and Monash University, Box Hill, Australia

Here, “Eastern Health” matches to https://ror.org/00vyyx863 and “Monash University” to https://ror.org/02bfwt286. Should the matching strategy return all the identifiers, one of them (if so, which one?), or nothing at all?

Similar questions arise when it is possible to match to multiple versions (or duplicates) in the target identifier set. This can happen, for example, in the context of bibliographic reference matching or preprint matching. Multiple matches may occur when there are different editions, reprints, or variations of the same publication in the target dataset, each with its own unique identifier.

If you are waiting for an answer to these questions, we unfortunately must disappoint you here. These can only be answered in the context of a specific problem, considering who the users are and what it is they need and expect.

Did you notice any other subtleties related to metadata matching and its concerns? Are there other non-obvious questions that should be considered when planning to develop or integrate metadata matching strategies? Let us know—we’d love to hear from you!

Ever since ROR was established, we’ve continually adjusted our resourcing and staffing to keep up with ROR’s needs and to align ROR activities with the needs and priorities of ROR’s operating organizations. The ROR team has therefore changed and grown over the years since the registry’s initial launch, evolving from an ad hoc project team to a more formal structure with dedicated staff members. This was made possible initially through startup funds from ROR’s operating organizations, contributions from community supporters, and multi-year grants.

As ROR began to move beyond its startup phase, ROR’s three operating organizations recognized the need to have long-term dedicated resourcing, and thus in 2022 they committed to supporting ROR’s core expenses on an ongoing basis. A shared staffing model is a critical part of this agreement, ensuring that ROR has dedicated personnel to carry out essential activities. Each organization essentially “contributes” one or more staff members to ROR on a dedicated basis, and we balance these contributions across our three organizations.

Staff arrivals and departures are an inevitable development in any team or organization, and ROR is no exception in this regard. Since the beginning of 2024, the ROR team has been experiencing a few changes – but at the same time, many things have stayed the same. We wanted to take this opportunity to update the community on how the team has been evolving.

In January of this year, long-time ROR Director Maria Gould left her position at California Digital Library to assume a new role as Director of Product at DataCite. With DataCite being one of ROR’s operating organizations, and ROR a core piece of DataCite’s overall strategic vision, it was natural for Maria to continue as Director of ROR, although in a slightly more limited capacity in order to fulfill the responsibilities of her new role.

As of this month, long-time Metadata Curation Lead Adam Buttrick has left his position at Crossref to fill the vacancy at CDL left by Maria. The CDL position is a senior-level Product Manager role focused on leading a portfolio of persistent identifier services, of which ROR is a key element. Like Maria’s time previously, Adam’s time will be split across ROR and other PID services and activities. With ROR’s metadata being a core product offering for CDL, Adam will continue to support metadata curation activities while also driving further product development work on ROR’s overall services and infrastructure.

Adam’s departure from Crossref means that there is a new vacancy to fill! ROR’s curation activities continue apace, receiving approximately 700 requests every month to add or update registry records and publishing new registry releases approximately every 2-4 weeks. Curation activities are also heavily focused on the incorporation of the Funder Registry into ROR, with ongoing reconciliation work to support all those who have switched - or are planning to switch - to using ROR for funding organization identification. As the scope of curation activities and product development has grown with wider adoption of ROR, we need to ensure we have adequate resourcing and staffing to keep up with these developments. We are now hiring a new ROR team member - Metadata Manager - at Crossref to coordinate ROR metadata curation as well as support related metadata initiatives at Crossref. Read the position announcement and please consider applying yourself or spreading the word to your networks!

All of these developments reflect how ROR continues to stabilize and mature, and they underscore the importance of continuing to grow the team to support ongoing needs as ROR evolves. At the same time, we fully intend to keep our operations small and nimble in order to stay focused on key objectives and remain flexible enough to adapt to future changes.

Photo by Jansen Yang on Unsplash

At Crossref and ROR, we develop and run processes that match metadata at scale, creating relationships between millions of entities in the scholarly record. Over the last few years, we’ve spent a lot of time diving into details about metadata matching strategies, evaluation, and integration. It is quite possibly our favourite thing to talk and write about! But sometimes it is good to step back and look at the problem from a wider perspective. In this blog, the first one in a series about metadata matching, we will cover the very basics of matching: what it is, how we do it, and why we devote so much effort to this problem.

What is metadata matching?

Would you be able to find the DOI for the work referenced in this citation?

Everitt, W. N., & Kalf, H. (2007). The Bessel differential equation and the Hankel transform. Journal of Computational and Applied Mathematics, 208(1), 3–19.

We bet you could! You might begin, for example, by pasting the whole citation, or only the title, into a search engine of your choice. This would probably return multiple results, which you would quickly skim. Then you might click on the links for a few of the top results, those that look promising. Some of the websites you visit might contain a DOI. Perhaps you would briefly compare the metadata provided on the website against what you see in the citation. If most of this information matches (see what we did there?), you would conclude that the DOI from that website is, in fact, the DOI for the cited paper.

Well done! You just performed metadata matching, specifically, bibliographic reference matching. Matching in general can be defined as the task or process of finding an identifier for an item based on its structured or unstructured “description” (in this case: finding a DOI of a cited article based on a citation string).

But matching doesn’t have to just be about citations and DOIs. There are many other instances of matching we can think of, for example:

• finding the ROR ID for an organisation based on an affiliation string,
• finding the ORCID ID for a researcher based on the person’s name and affiliation,
• finding the ROR ID for a funder based on the acknowledgements section of a research paper,
• finding the grant DOI based on an award number and a funder name.

Matching doesn’t have to be done manually. It is possible to develop fully automated strategies for metadata matching and employ them at scale. It is also possible to use a hybrid approach, where automated strategies assist users by providing suggestions.

Developing automated matching strategies is not a trivial task, and if we want to do it right, it takes a great deal of time and effort. This brings us to our next question: is it worth it?

Why do we need matching?

In short, metadata matching gives us a more complete picture of the research nexus by discovering missing relationships between various entities within and throughout the scholarly record:

These relationships are very powerful. They provide important context for any entity, whether it is a research output, a funder, a research institution, or an author. Imagine for a moment the scholarly record without any such relationships, where all bibliographic references, affiliations (institution names and addresses), and funding information (funder names and grant titles) are provided as unstructured strings only. In such a world, how would you calculate the number of times a particular research paper was cited? How would you get a list of research outputs supported by a specific funder? It would be incredibly challenging to navigate, summarise, and describe research activities, especially considering the scale. Thankfully, these and many other questions can be answered thanks to metadata matching that discovers relationships between entities in the scholarly record.

There are two primary ways we can use metadata matching in our workflows: as semi-automated tools that help users look up the appropriate identifiers or as fully automated processes that enrich the metadata in various scholarly databases.

The first approach is quite similar to the example we described at the beginning. If you are submitting scholarly metadata, for example of a new article to be published, you can use metadata matching to look up identifiers for the various entities and include these identifiers in the submission. For example, with the help of metadata matching, instead of submitting citation strings, you could provide the DOIs for works cited in the paper and instead of the name and address of your organisation, you could provide its ROR ID. To make this easier for people, metadata submission systems and applications sometimes integrate metadata matching tools into user interfaces.

The second approach allows large, existing sources of scholarly metadata to be enriched with identifiers in a fully automated way. For example, we can match affiliation strings to ROR IDs using a combination of machine learning models and ROR’s default matching service, effectively adding more relationships between people and organisations. We can also compare journal articles and preprints metadata in the Crossref database by calculating similarity scores for titles, authors, and years of publication to match them with each other and provide more relationships between preprints and journal articles. This automated enrichment can be done at any point in time, even after research outputs have been formally published.

There are fundamental differences between these two approaches. The first is done under the supervision of a user, and for the second, the matching strategy makes all the decisions autonomously. As a result, the first approach will typically (although not always) result in better quality matches. By contrast, the second approach is much faster, generally less expensive, and scales to even very large data sources.

In the end, no matter what approach is used, the goal is to achieve a more complete accounting of the relationships between entities in the scholarly record.

This blog is the first one in a series about metadata matching. In the coming weeks, we will cover more detail about the product features related to metadata matching, explain why metadata matching is not a trivial problem, and share how we can develop, assess, compare, and choose matching strategies. Stay tuned!

Today marks a red-letter (teal-letter?) day for ROR: the official public launch of version 2 of the ROR schema and API! Version 2 of ROR, developed and beta-tested over many months with the generous input and help of the ROR community, is more streamlined and more powerful than version 1, including some restructuring that will make ROR data easier to manage and use and some new fields that ROR users have been asking for. Read on to find out more.

What’s new - summary

Here’s a synopsis of what’s new and improved.

• We’ve added a field for administrative information that includes the date a record was created and the date it was last modified. This crucial change enables users to retrieve only the most recently added and updated records from the ROR registry, which has been a much-requested feature.

• We’ve simplified name information and have made changes to ensure that organization names in ROR work for a global, multilingual community of users. By adding a place to store language codes for all name variants and treating names in different languages as equivalent, we’ve made ROR metadata less Anglocentric and more useful for countries with multiple official languages.

• We’ve removed redundant and overly granular location information, keeping only the most useful and globally relevant fields: country name, country code, latitude, longitude, specific location (city), and GeoNames ID. These changes make location information much easier to read, understand, and work with.

• We’ve made organization website information more useful by adding a field for an organization’s web domains, creating a single container field for the organization’s website and its Wikipedia page, and removing deprecated fields for IP address and email address that were inherited from GRID and never populated. Web-related metadata is now more compact and better organized, and users will now be able to disambiguate organizations by domain.

• We’ve restructured external identifier information and made it more flexible in order to make it easier to manage current and potential future mappings to other organization identifiers.

• We’re adding an additional organization type, “funder,” which can be combined with existing organization types in ROR such as “nonprofit” and “government.” With ROR set to become the standard identifier for funders, this change will make it easier for ROR users to recognize and select funding organizations.

• The ROR API now exists in two parallel versions, and we’ve made sure not to break any code that current ROR API users have written. Version 1 of the ROR API is now available at https://api.ror.org/v1/organizations and version 2 of the ROR API is now available at https://api.ror.org/v2/organizations, but version 1 is still available at the original location without the version in the path: https://api.ror.org/organizations. Version 1 of the ROR API will remain the default for at least another year, through April 2025 and perhaps beyond.

• All the same records will always be available at the same time in both versions of the API, and all the same functionality of version 1 of the ROR API remains available in version 2. Users of version 2 of the ROR API, however, will be able to send additional queries based on version 2 metadata, such as queries based on created and last modified dates.

• The ROR data dump now includes JSON and CSV files formatted according to both schema v1 and schema v2. This means that there will now be 4 files in each data release instead of 2, and users can choose to use the format and schema that best suits their needs. Take a look for yourself by downloading the most recent ROR data dump from Zenodo at https://zenodo.org/doi/10.5281/zenodo.6347574.

• The ROR web search at https://ror.org/search now uses version 2 of the ROR API, and we’ve taken the opportunity to make a few improvements. Organization names are now identified by type and language so that it’s more apparent what’s an alias, what’s an acronym, and what’s in an alternate script. Additionally, each individual landing page for a ROR record now includes a link to the JSON view of the record, which enables users to see the underlying data behind the interface.

Compare a v1 record with a v2 record

Compare the ROR record for Victoria University in ROR v1 with the ROR record for Victoria University in ROR v2 to see the improvements we’ve made in the metadata.

Victoria University, ROR v1

Victoria University, ROR v2

What’s new - more details

If you’d like to see even more details about what’s new in version 2 of the ROR API and schema, take a look at our changelog and at our v2 documentation. In our documentation of version 2 of ROR you’ll find all the new v2 fields and subfields listed by name, sample v2 API queries and results, and fully updated guides to implementing ROR using version 2. You can easily toggle between v1 and v2 documentation.

Thanks to all who made this possible!

The launch of version 2 of ROR ranks right up there with other major milestones for ROR, including the stakeholder meeting in Girona, Spain in January 2018; the “Minimum Viable registry” launch in January 2019; and the first independently-curated release in March 2022. We’re very grateful to all those who made this watershed moment possible, including all the members of the ROR community who gave us feedback on our proposals and participated in our beta test.

If you’d like to participate in future ROR developments, please get involved with the ROR community!

• Subscribe to the quarterly ROR newsletter
• Sign up for the ROR Technical Forum
• Join the ROR Slack
• Attend a ROR community call or other event

We had over 300 unique registrants for our three recorded sessions, making this the most popular ROR annual community meeting ever! It was a true delight to see so many ROR users and supporters come together and to meet some people who are new to ROR. If you missed the meeting, here are the highlights.

ROR Annual Meeting and Fifth Anniversary Celebration

The 2024 edition of the main ROR annual meeting was a special one! We took the opportunity to reflect on the last five years, remembering where we started and how we got here. We heard about where we’ve been and where we’re going in terms of our community of adopters, our curation activities, and our tech. Most importantly, we acknowledged ROR community contributors, without whom we’d never have come this far, and we did some brainstorming about what’s next.

Selected highlights

• A look back at memories of ROR from 2016-2019, including a white paper on the concept of an organization identifier registry, a working group report, and pictures from in-person events
• Stats of ROR adoption include a staggering 1.48 million DOIs registered with DataCite that include ROR IDs to identify creators or contributors
• The number of requests to add new records to ROR has surged from 50 per month in January 2022 to more than 700 per month in January 2024
• Key technical accomplishments in 2023 include schema v2 development, implementation, and beta test; anticipated for 2024 is the public launch of v2 and a redesigned affiliation matching service
• ROR Community members are in strong alignment that we are on the right track and should keep going, specifically focusing on continuing our work on publisher adoption and the Crossref Open Funder Registry transition in 2024

Community Showcase

The Community Showcase featured presentations on how ROR is used in some key global scholarly systems and workflows, including at OpenAlex, 4Science / DSpace-CRIS, ORCID, Springer Nature, US Department of Energy’s Office of Scientific and Technical Information (OSTI), and the ROR Implementation Task Group of the NIH’s Generalist Repository Ecosystem Initiative (GREI).

Selected highlights

• A live demo of OpenAlex’s web interface, whose institutional data relies on ROR
• ROR will be on the roadmap for DSpace 8.0
• ROR IDs now account for half of new affiliations created in ORCID; over 1.2 million ORCID records have a ROR ID and more than 50,000 unique ROR IDs are represented in ORCID
• Springer Nature currently has over 1.1 million ROR IDs in its metadata and its next steps are to add ROR IDs to already published content, provide ROR IDs to Crossref during metadata deposit, and evaluate other use cases for ROR
• OSTI will begin including ROR IDs in its metadata once they deploy their new internal authority as part of E-Link 2.0, their new research submission tool
• Of the 7 repository systems participating in NIH’s GREI initiative, 5 have already adopted ROR for organizational affiliations and 3 have adopted ROR for funder associations

Why We All Need Good Funding Metadata

The session Why We All Need Good Funding Metadata addressed the issue that funding information is an increasingly important piece of metadata for many stakeholders, yet standardizing and using identifiers for funders remains a challenge. With ROR set to become the standard persistent identifier for funders, we convened a panel for ROR’s annual community meeting to hear diverse perspectives about how funding metadata is used and what might make it better. We heard from funders, publishers, metadata specialists, and more, including CHORUS, the Royal Society of Chemistry, Dryad, Crossref, and Stratos for Aligning Science Across Parkinson’s.

Selected highlights

All panelists were asked “If you had a magic wand, what would be the one thing you’d do to improve funding metadata?” The answers:

• “Make extracting PIDs and their associated metadata from articles, datasets and software simple, open and affordable for all!”
• “All organisations and systems consistently using standard PIDs for people, organisations, grants, projects, data and publications throughout the funding, research and publication processes”
• “Systems/tools can accurately extract & identify funders from authors’ acknowledgements and match to funder IDs at the point of submission”
• “All funder data available through APIs”

Let’s ROR some more in 2024!

ROR has come a long way in the last five years, and ROR is going even farther in the next five. Heartfelt gratitude is due to everyone in the community who has helped to make ROR a reality, including the ROR Operations Team; the ROR Curation Advisors; the ROR Steering Group; ROR Sustaining Supporters; ROR adopters and integrators; members of the ROR technical forum and ROR Slack; subscribers to the ROR quarterly newsletter; our colleagues at Crossref, DataCite, and the California Digital Library; all the alumni and affiliates who helped build ROR; everyone who presents at a ROR community call and participates in our case studies; all those who attend ROR events; and the many, many people who ask for a change or addition to ROR records. We wouldn’t be here without your engagement and support!

Hope to see you all throughout 2024 and at the next annual meeting in 2025.

Amanda French

Thank you so much for doing this with us. Can you please tell us your name, title, and organization?

Cameron Neylon

I’m Cameron Neylon, Professor of Research Communications at Curtin University, in the School of Media, Creative Arts and Social Inquiry, which is as mixed a school as it sounds like. We’ve got everything from social sciences to critical literary studies to people doing painting and sculpture and journalists, so it’s a really mixed and diverse group. And then within that, I’m co-lead with Professor Lucy Montgomery of the Curtin Open Knowledge Initiative.

Amanda French

Tell us about the Curtin Open Knowledge Initiative. When did that get started and why?

Cameron Neylon

COKI, as we call it, started probably in 2015 or 2016. Lucy and I both returned to Australia after having been in Europe and overseas, and we found the level of conversation about open scholarship in Australia to be quite backwards, and behind, and kind of frustrating. And it was particularly frustrating because it was clear that this wasn’t for a lack of interest or desire to move things forward: it was in large part because the leadership of universities weren’t engaging with these issues, because none of the information resources that they were using brought this up as an issue.

Australia is and has been particularly obsessed with university rankings, partly because the sector is very dependent on international students as a revenue source. But as we know, those rankings are incredibly narrow, backwards looking, conservative, opaque, and actually not very informative. And they certainly don’t do anything to get a university to address questions of open access, data sharing, staff demographic diversity, or many of the kinds of issues that are bound up in this idea of open knowledge or open research.

What we eventually realized was that there’s this gap in knowledge, this gap in understanding, and this gap in information resources. So the idea was to do a better job of providing actionable information in the form that universities are expecting to see it, a form that shows progress towards aspects of open knowledge and comparisons with other institutions, and then that would be one way of provoking change.

We were very lucky to pitch this to a receptive Deputy Vice Chancellor for Research who had just started, and as a result, we got a lot of support from the university to take this forward. That’s all very abstract, of course. What we actually started to build was a collection of open data about institutional practice, about research outputs, open access, citations, traditional kinds of things, but also things like staff demographics and other kinds of information that we thought were relevant to this big picture.

We were very dependent on the growing set of open data resources that were developing, have continued to develop, and will continue to develop. And we got lucky in our timing, I guess, in the sense that we started at a point where those open data resources were just about good enough to compete with proprietary data sources. But they’re accelerating and improving at such a rate that now they are vastly superior, in almost all cases.

That and various advances in cloud infrastructure and computing allowed us to bring all of these datasets together, and so with the goal of making the information actionable, we essentially brought the big open datasets together into a large database using cloud infrastructure. Then we set about building things that we thought might be useful. And as I said, we had the financial support and resources that gave us the luxury to do that and let us do it properly rather than doing small projects. That has let us build up this immense data resource, which is really powerful, but it has also let us build some of the kinds of information resources that we think are needed to help people make intelligent decisions about change.

Amanda French

Amazing. So when and where did you personally first hear about ROR?

Cameron Neylon

That’s kind of hard to pin down. I have been invested for a long time in the idea that persistent identifiers are central to making use of open information at scale, and we’ve learned that over and over again. When we started COKI, it was just before ROR existed, although there were conversations happening about heading in that direction. But Digital Science had made the GRID database available under an open license, so we were happy to be able to use that as a source of information for indexing around institutions. And of course, the GRID IDs were used by Microsoft Academic. And Microsoft Academic was one of our central data sources.

Then, as ROR developed, first as a concept and then as a reality, we were watching that whole journey, waiting for the moment when the maturity of the systems was there to jump across, because we obviously preferred something that was organized through community infrastructures. The thing that really pushed us across in the end, of course, was Microsoft Academic shutting down and OpenAlex stepping into that gap, specifically. That coincided with a range of things we were doing. But actually, in the end, with all of the architecture and systems we had to change and the move from Microsoft Academic to OpenAlex, the shift to ROR was one of the easiest ones. In fact, we just dropped the entire ROR dataset into an identifier column. It was very simple.

Amanda French

That’s great to hear. I’m talking to an increasing number of people who are using that OpenAlex dataset, and thereby using ROR, because ROR is currently the only external institutional identifier that they use.

Cameron Neylon

What’s central in our work is that we bring all these datasets together. We have OpenAlex, ROR, Crossref, Crossref Event Data, Open Citations, OpenAire, PubMed, and I’m probably missing a few. We bring all of these datasets together, and we organize them around community governance and persistent identifiers. And rather than just saying, “Okay, what can OpenAlex tell me about DOIs connected with this institution as identified by ROR?” we can do that across multiple datasets. At the moment, that works most effectively for DOIs, because the DOI has the widest uptake as a standardized identifier across these systems.

But that also points us to what’s really powerful in terms of being able to connect these datasets up. Increasingly, the first step we take with integrating a new dataset is if it doesn’t have those standardized, persistent identifiers in it, we do the mapping across, because the dataset is just not very useful without it. PubMed is a good example of this; I was working with it the other day. It has a bunch of institutional identifiers, but actually mostly the affiliation information in PubMed appears to be in the form of strings. And we just can’t leverage that dataset as well as we might for organizational information, because those links aren’t there. We can leverage it really well in terms of the publications, and increasingly for datasets as well, because we’ve got the DOIs. But for organizational information, it’s just that much weaker, so things like getting all the information for funding decisions is a lot harder than it needs to be.

Amanda French

Are there any other entities that you wish were using ROR IDs more?

Cameron Neylon

Oh, I mean, everyone. It’s really difficult to describe clearly just how much of a difference it makes. And I think that we’re rapidly moving towards a world where this is going to become self-evident. In the old days, if you wanted to ask a question, you first started with, “Okay, which dataset should I use to answer this question, and what limitations does that dataset impose on me?” With open data and cloud infrastructures and the availability of these datasets as a whole, that’s not where we start to answer a question. We ask, “Which datasets can we bring together to maximize the amount of information and the precision that we can apply to addressing this particular question or problem?” The datasets that are easy to integrate are the ones that we use. Increasingly, if there isn’t a ROR ID or the ORCIDs aren’t properly handled, then we just don’t use the datasets.

The other side of that is because we are operating at a fairly large scale and making these public information products, we get a lot of people coming back and saying, “Oh, you’re missing this,” or “That’s not there,” or “There’s something wrong here.” We can feed that back, and we can feed it back really easily. An example of this exactly with ROR: there was a university and the librarians noticed on the global open access website that the counts of outputs seemed a bit down. They looked closely at name variants, and they discovered that there was a bookstore on campus, and OpenAlex had assigned a bunch of the outputs from that university to the bookstore. OpenAlex is still a work in progress, so that kind of thing does happen. And the bookstore wasn’t listed in the hierarchy for the institution; if that had been the case, everything would have flowed up nicely.

What that story illustrates is that we don’t find these errors. The people working with the data at scale, making information products, we don’t often find those kinds of errors. They’re always found by people on the ground who know the context and know exactly what the numbers should look like. We get a couple of emails a week from various people saying “This is wrong,” “This is under-counting,” “This is over-counting,” “These two organizations don’t belong together,” or “These two organizations do belong together.” And it’s really easy for us to pass that information back up when there are good identifiers. And we can say, “No, this DOI should link to this ROR,” or “This ROR should probably be a child of that ROR.”

That flow of curation information into the system, combined with the ease of use when things are easy to interconnect, I think is ultimately going to make these little, isolated, increasingly fragmentary datasets that are difficult to work with because they don’t have the coverage and they don’t have the interconnection less relevant in the long term.

Amanda French

Fascinating. Well, I hope you and your staff do refer those kinds of issues to ROR, because we are always happy to make those kinds of corrections. If there is a bookstore that should or should not be in the relationships of a ROR record, we’re happy to add that or take it out.

Cameron Neylon

It raises lots of really important questions about the importance of community governance. In these cases, a lot of the time, when someone tells us something’s wrong, sometimes that’s a mistake we’ve made, and we can correct that. Sometimes it’s a mistake in someone else’s data system, and then we can pass that on and explain it to them. And sometimes it’s a thing where there’s reasonable grounds for disagreement.

And in fact, what we said in this case was, “It’s not our place to say whether that bookstore is part of that university. That’s a decision you guys have to make, and here’s the ROR request form for when you figure that out.” And I think that’s a really important part of the whole story of changing the way people think about corrections.

In the space of bibliometrics and scientometrics, almost every university in the world has people whose job it is to correct the things that are wrong in Web of Science and Scopus. This is like an extra full-time person on top of the hundreds of thousands of dollars that are spent on a subscription to these products, working to help correct them. But that data just goes into a black hole, which we can’t reuse.

And with this broader open data ecosystem, if people feel responsible for contributing to collectively raising the quality of it and working with these community infrastructures, then the capacity for improving the data for everyone, so that everyone’s data looks better, is just huge. And it’s sometimes a challenge, because people are not used to feeling like they have authority. In many cases, data was always something that was done to them. And yet this is data about the stuff we are doing, and the organizations that we are. I think it’s a really interesting transition to try and navigate, to get everyone get more engaged in feeling like they have a stake in making these things better.

Amanda French

It is interesting too, because ROR’s curation model, as I’m sure you know, is not the same as something like Wikipedia or Wikidata, in the sense that we curate everything. Anyone can request anything, but we evaluate that and approve or decline it centrally. Wikidata and Wikipedia have a decentralized, distributed editing model, whereas we have quite a centralized curation model.

But we also don’t have a model in which only certain people can make requests, or where there is a particular person at every organization who is responsible for making sure that their information is correct in ROR. We will take a request from anybody; you don’t have to be affiliated with that organization. And we evaluate that in the open based on openly available information and on what makes sense for ROR.

And I think that model is unusual for people to understand. I think they tend to have a kind of a binary notion of authority over data. Either there’s a centralized authority who is making decisions, and you are lucky if you have any input into that, or it’s something that’s entirely distributed. I really like our model, because I do think it has the best of both worlds. We will entertain any corrections that you wish to give us, but we do also review and curate that quite carefully.

Cameron Neylon

I think I would agree, in the sense that there’s clearly a tension here, right, between consistency of application and the scale at which the work can be done. With federated models, people assume that what you might lose in precision you gain by having this massive scale of work that can be done. But what you also do is create a massive scale of work that needs correction. It works, kind of, in some parts of Wikipedia or Wikidata, but mainly because of the absolutely vast scale of those enterprises. And anyone who’s done anything on crowdsourcing knows that it’s really, really hard to replicate that level of success at a smaller scale, so the tension there of needing to have a reasonable amount of effort and a reasonable application of resources but still get something out that’s usable drives you toward the other end of that spectrum, toward centralization.

And indeed, I think that’s again where the community governance and the model of transparency comes in. And it’s got to be flexible, not rigid, because the other assumption people make is, “Well, surely you can automate this or have some sort of system that just does all the checking for you.” AI, wave hands, wave hands. And actually, none of those things work, in part because there are bad actors in the system, and you’ve got to be on the lookout for that.

That’s particularly true in terms of organizations and who they are and what they’re connected with and who’s connected with them. Those are exactly the areas where we’ve suffered a lot of fraud in the scholarly system, so that’s got to be managed in some way that’s ideally flexible and transparent, but also trustworthy. At the end of the day, that’s where you’ve got to be going, otherwise these things just aren’t usable.

Amanda French

Can I ask, do you teach?

Cameron Neylon

I don’t do a whole lot of teaching. My role is mainly research, at least at the moment. Our work doesn’t neatly fit into a teaching program in the school that we’re within. Like a lot of multidisciplinary groups, there are challenges in terms of where we are and what we’re teaching. It’s a peculiarity of the Australian system that there is a tendency to separate research and teaching, and that’s driven by the financial model that the government imposes on Australian universities more than anything else. So, yes, I think there’s always some challenges around that.

That said, we are in the school that contains the Information and Library Science program, and so we do teach into that, and then more broadly, into what you might call Critical Information Studies. If you’re over in the sciences, you’d probably call it Data Science and have a little bit of ethics and critical theory applied. If you imagine the other side of that, we’re starting from critical perspectives about what information is and what information does and whose benefit it serves, and then teaching a bit of the technical side so that people get an understanding of how that works out in practice: what the nitty-gritty of actually doing data analysis and visualization looks like, and why the tools are not neutral, and why it often looks like the entire world’s data has been shoved through an Excel spreadsheet, because the limitations of Excel spreadsheets are what drive the things people do with data.

Amanda French

Ha! Right. The reason I asked is, I imagine you have fellows or postdocs or graduate students or whatnot working with you on your projects, and I’m just curious how you address persistent identifiers when you’re passing knowledge along to the next generation, whether that’s in a classroom setting or in an apprenticeship-slash-lab setting.

Cameron Neylon

That’s a good question. I would start by observing that we’ve probably still got some work to do with the current and indeed past generations on the use of persistent identifiers.

Where we do teach it is we have a team, we bring people into that team, and where we educate people about this is through the practice. We have processes by which we write up documents and put them by default on Zenodo in a COKI Zenodo community so they get a DOI and so they can be automatically connected to people’s ORCIDs. We use the Tenzing tool to apply CRediT taxonomy when we prepare things, or at least we try to do that as much as possible.

At the social level, then, if you like, we try and embed it as much as possible into what we do and how we do it. And then at the technical level – and most of our team is pretty technical – it also goes into the sort of architectural principles we have for our data structures. In more traditional teaching, we teach some courses around data and telling stories with data.

And actually, introducing the concept of persistent identifiers and why they matter is always a challenge. Do you present it in kind of an abstract and theoretical sense of how well the system could work? That doesn’t really speak to very many people. Or do you present it in a much more practical and concrete way? There, you run up against the fact that at least until recently many of these systems just didn’t work as well as they might.

I’m always reminded of the person who, when being asked to sort out their ORCID record, upon being told this was the last time they’d need to sort out a personal profile as a researcher, observed “Well, that was what I was told the last five times I had to do a personal profile as a researcher, and every one of those has fallen by the wayside, and now you want me to do another one, so how can you promise me another one’s not going to come along next week?” One of the big challenges with all of these systems is they will work really well when everyone’s using them and when everyone doesn’t even think about using them, but getting there is hard work.

Amanda French

Yes. And the metaphor I always think of, which is possibly a bit too simplistic, is the telephone. You have telephone numbers, and everyone needs a unique telephone number, and telephones themselves increase in utility the more people have them. And the more people who have telephones, the more important it is that you have unique telephone numbers. And if I were in a research position at a university, I would probably start doing all kinds of research into party lines and alphanumeric exchanges and the emergence of cell phones and things like that, you know, looking at the history of how we got to the globally standardized system that we have today for telephone numbers.

Cameron Neylon

And I guess the challenge with that, thinking about this, is that if you take that analogy through to the current world of social media or streaming video platforms, everything’s so vertically integrated that I don’t know if very many people are aware of why it is that a phone number can be a standard thing, despite the fact that lots of different people make phones and lots of different carriers will carry phone calls. And similarly with email, because those things are the last residue of that in the world as a whole.

Most of the things we deal with don’t have those standards, and that’s why they’re awful, right? It’s why you’ve got three streaming subscriptions, not one TV aerial, and why you’ve got seventeen different direct message or personal messenger accounts across multiple different services, none of which you really like to use, but because there’s one person who does like to use that particular one you have to have it. And you know, the seemingly obvious concept that you should be able to get those messages wherever you want to, and send them to anyone based on a standardized sort of protocol or system, has partly been lost. And that’s part of the battle of making the case for how these things can work better, is, yes, recalling the fact that you can in fact make phone calls and you don’t have to have an AT&T phone and a British Telecom phone.

Amanda French

That’s right. I think that’s such a good point about looking at the failures of standardization, and the takeover of non-open commercial services for these things. I suppose some other examples would be power systems, globally. You know, when you travel from country to country, the plugs are different in the United States than they are in Europe, and, as I discovered recently, also different in South Africa. Measurement systems, metric versus non-metric, not standard everywhere you go. Even driving: which side of the road you drive on and which side of the car the steering wheel is on. All of those things are small frictions, and yet they’ve been standardized enough to where they work regionally, and we can see how well it would work if they were standardized globally.

Time zones, actually! Obviously we have to have time zones, just because of the fact of having a globe, but I keep thinking, “Wouldn’t it be nice if everybody got rid of Daylight Saving so that we didn’t have the problem of wondering if such and such a country has gone on Daylight Saving?” If it were a standard interval between each timezone and didn’t change with the seasons it would make scheduling international meetings so much easier.

Cameron Neylon

And also the weird time zones, as well. The power one is actually quite an interesting one, because there does seem to be an emerging plug standard. There are actually moves towards persuading everyone to just use one. I mean, it’s a bit ridiculous, but given that a lot of devices that we use, with some exceptions, can run off 5 volts, an awful lot of things could just use USB plugs. There’s a 5 volt / 12 volt / 24 volt issue, but a lot of things could use USB, and we’re sort of iterating towards that. Interestingly, it’s driven not really by thoughtful top-down standardization, but by frustration of people who are traveling and the bag full of adapters that you’re carrying, if you’re going to more than one place. I do actually carry around now just one or two USB adapters with swappable plugs, because that works reasonably well. Until I get to South Africa and realize that that particular plug is quite unusual.

But yes, I think there’s something really interesting there. The other example from the power world, of course, is that once upon a time you couldn’t take any appliance from, say, the US at 110 volts and bring it to anywhere in the world running at 240 volts. But these days, I don’t even remember that I should probably check before I plug something in with an adapter, because the transformers are all standardized to deal with both cases, so you don’t usually have to think about needing a step down transformer to operate.

And that gets back to things like ROR as interchanges, where yes, there are legacy systems of other identifier systems out there, and there are systems that perhaps ROR shouldn’t cover that maybe ISNI needs to, but at least if we’ve got places to do lookups, and those are, again, centrally maintained and kept reasonably up to date, then we don’t need to think too much about that as we translate between things.

What I wish we also had, and ROR does this well, and other identifier systems not as well yet, is exactly this thing of collections of objects. Most of the work we do involves taking sets of research outputs and bundling them up into groups where those groups might be, they might be the outputs of a person, they might be the outputs of a project or an institution or a country or a discipline, or whatever it might be. And in the institution case, where because of the parent-child relationships in the ROR schema, we can go up and down those hierarchies and figure out the level we want to operate at. Whereas I’d say with DOIs, that’s not so easy, because there’s a lot of complexity and inconsistency in relationships between DOIs for collections versus DOIs for objects. And that’s before I start getting annoyed about ISBNs and book chapters.

Project IDs, I think, are an area that’s missing a lot of space, and I think institutions really ought to do a much better job of providing ways of interconnecting between ORCIDs and institutions. There’s lots of opportunities there. And thinking carefully about those collections kind of questions, I think, is really valuable.

Amanda French

Maybe that’s a good occasion for me to ask, what do you hope ROR does better in the future? What can we do that would make your life easier?

Cameron Neylon

There’s very little that we bump up against as a problem. At the moment, I think most of the immediate issues we’re facing are to do with the challenges of actually making the interconnections, so the assignments of, say, outputs to two organizations, and the heuristics involved with that, which is a hard problem to solve.

The area where I can imagine the real benefits will arise, and this is already happening, is the expansion of the set of organizations being covered, and that’s really welcome. When there’s a new release, that number clicks up and you think, “Oh, my brain is about three releases behind, I thought it was still 95,000, and now it’s 110, 115, 120, something like that.” [Note: It’s currently over 107,000.] I think that’s great. I’m really glad that people are thinking hard about how to manage those questions of curation, not because I have opinions necessarily about how they should be done better, but because I know that they are really hard problems. I’m glad smart people are thinking about them.

One of the questions I have, and it’s an issue that, again, is a real challenge, both from a curation perspective and from a schema perspective, is how we can tackle the question of units within organizations and what that looks like. I’ve seen that on both sides. I completely understand why people would want to run screaming in the opposite direction from even trying to deal with it. And I can also see from within institutions how important it is to have the capacity to be able to do that. I think that’s an area that’s really worth thinking through. And the answer might be something like component DOIs, so actually for ROR to not deal with it, but to allow institutions to self-manage. I can also see that going horribly wrong.

Amanda French

Right.

Cameron Neylon

I’m not sure I’ve got a better solution. But one of the things we see really frequently is exactly that question of how to reflect back to an institution what the parts of it are looking like while having to deal with the fact that only the institution itself has any sort of real knowledge of what that looks like. And it’s changing so fast!

Amanda French

Yes, I know. Departments, in particular, will merge and change names and whatnot. And I have to tell you, I absolutely believe that there would be internal arguments at an institution about what their institutional hierarchy would be.

Cameron Neylon

These are real issues, right? This is the thing we discovered, I mean, coming right back to the starting point of why we were doing this in the way we did. What a Vice-Chancellor or the University President sees is something like “Here are the faculties, here are the number of outputs, here are the number the citations, here’s the amount of research income.” It’s really at a very high level. And those numbers are garbage for the most part. They are appallingly bad. Years out of date, and often related to an organizational structure that hasn’t existed for half a decade or more.

And so there’s a desperate need for automating more of this so that information will be more up-to-date and more responsive and more flexible. But at the same time, if heads of schools are being judged on how much research revenue they’re getting assigned to them, and they’re being judged on how much they’re collaborating across the university, are they going to quietly not argue about where that money got assigned to, even if it then got spread out across multiple faculties or departments?

It’s a real issue on which people’s livelihoods depend. They’re really complicated questions, and getting them wrong can break institutions. Information has to be summarized, and summarization is a lossy process, and so you’ve got to make decisions about what you lose and what matters.

And maybe that’s the other piece, and I think this isn’t just a challenge for ROR, but the question of literacy. How do we raise the literacy of all the players? Not to the extent that they need to know the nitty-gritty of all the details of how everything works, but so they can ask intelligent questions about how the information that they’re getting has been processed, and whether at least in principle there’s a transparent audit trail so they can go back and check and see how things might have looked if it was done differently. And again, that’s something we build into pretty much everything we do now.

It’s amazingly common in the proprietary world not to do this. University rankings are particularly bad at this. The thing you do if you’re going to change something is you do one run the old way, and then you do your next run the new way, and you look at how different they are. And you make that clear and transparent, and ideally, you’ve got open source code. It’s critically important, but it’s amazing how rarely it’s done, and even where it is done, without community-owned open indexes and infrastructures and systems, it’s really hard for that to be for that to be transparent.

These are the real challenges. People are exercised about artificial intelligence, and this, that, and the other, but the problems aren’t really to do with the scale at which these computational systems can work. They’re to do with the fact that people are making decisions so far away from the point where the context and on-the-ground information is that the quality of the information that’s going through those systems needs to be a lot higher than it is.

And the consequences of getting it wrong is that entire departments or even entire universities get closed down. Whole programs of research cease to exist. Teaching programs that you might need tomorrow are axed, because apparently someone didn’t get a job five years ago. Every time there’s a major world event, there’s a story that goes round about how someone three days earlier didn’t get a grant about that particular thing, because it wasn’t important. And suddenly it is.

Amanda French

Yes, absolutely. Well, we can wrap up. What else would you like to say about ROR or about your work?

Cameron Neylon

At core what I would say is we couldn’t do what we do without ROR. We’re focused on organizations and institutions. Without the underlying infrastructure that ROR provides to help us triage data from multiple data sources and connect it to organizations at the right level, we couldn’t do what we do. And as we move forward with what we’re trying to do, and try and scale up and scale out the kind of stuff we’re doing, it’s great to see that ROR is quite frequently a couple of steps ahead. ROR is thinking about the same problems and trying to address them as we can see them coming down the road ahead of us. So, yes: keep on keeping on.

Amanda French

We will! We’re feeling good. We are hiring more curator help and thinking about ways to scale up that curation that you noticed is so crucial. This has been really marvelous. I only wish we could talk for another half an hour about infrastructure generally. Thanks so much.

Cameron Neylon

Have a good day.

Looking back on 2023, we wanted to highlight a selection of milestones from the past year that we are especially proud of and that particularly represent ROR’s unique journey so far.

ROR as a new solution for identifying funders and addressing key use cases for funding metadata

In September, Crossref and ROR announced that the Open Funder Registry would be merged into ROR. With a significant number of Funder IDs already represented in the registry, ROR is well positioned to incorporate the Funder Registry and be used in research and publishing systems to identify funding organizations. Users are beginning to switch from the Funder Registry to ROR, and we’ll be sharing more updates on this progress as the transition continues into the new year.

Notable ROR integrations announced, updated, and completed

ROR adoption in publishing platforms continues to grow, with recent developments in AAAS’s Science Content Tracking System, Octopus, Janeway, ResearchEquals, and ScienceOpen. This year saw more integrations in repository and CRIS systems such as figshare for Institutions, ChemrXiv, DSpace-CRIS, InvenioRDM, and Zenodo; and in many other tools and indexes such as the COKI Open Access Dashboard and OA.Report. We’re looking forward to hearing about and announcing even more ROR integrations in 2024. Throughout the year, the ROR blog featured in-depth conversations with ROR adopters about how and why they are using ROR to address key needs in their workflows. Check out the complete list of case studies, and let ROR know if you have suggestions for upcoming features.

Development of version 2 of the ROR schema and API

Following extensive community input, ROR developed the first major update to its schema and API (v2), which will incorporate key feedback from users and integrators and enable ROR to build upon the data model originally inherited from GRID. The beta version of the new schema was released in September, and the production release is expected early in the new year.

Enhancing ROR’s global coverage and metadata quality

This year marked significant growth in ROR’s community-based curation model and included a number of dedicated efforts to improve the registry’s coverage in specific areas. These efforts included–among many others–a collaboration with France’s Ministère de l’enseignement supérieur et de la recherche focused on representation of French research organizations and their research units, comprehensive reviews of Latin American and Caribbean organization records by members of LA Referencia, and analyses of Spanish organizations by both FECYT and CSIC have contributed to enhancing the data quality and coverage for the country. In the United States, collaborations with government organizations such as the U.S. Department of Energy Office of Scientific and Technical Information, NOAA, and the Defense Technical Information Center have ensured more comprehensive and accurate representation of US government organizations. We are grateful for the numerous contributions provided by ROR’s community, which have greatly enhanced the registry’s value and utility.

Participation in a range of community events

In 2023 we joined community members in person and online at the Society for Scholarly Publishing annual meeting, the #PIDsLATAM workshop in Buenos Aires, a workshop at the Open Repositories conference, the biannual meeting of the International Society for Scientometrics and Informetrics (ISSI), and a number of webinars with collaborators including ORCID, DataCite, AfricArXiv, and DOE/OSTI.

We will be kicking off the new year with a celebration of ROR’s fifth anniversary at the end of January. Every year since ROR’s launch, we have celebrated the anniversary with an open community meeting to reflect on the past and plan for the future. The 2024 meeting will be an opportunity to mark five years of ROR and bring the community together to discuss the next five years and beyond. Stay tuned for more information about the celebration, and in the meantime please reserve the slots on your calendar by registering at the links on ROR’s Events page.

ROR wouldn’t be ROR without the community behind it. We express our sincere gratitude and appreciation for the many ways in which community members around the world have supported and engaged with ROR over the past year: showing up to community calls, giving us feedback on ROR’s technical development work, submitting registry updates, building ROR implementations, making financial contributions, and much much more.

Here’s to 2023, and to ROR-ing more in 2024!

Amanda French

Thank you so much for agreeing to talk to me about ROR. Can you begin by telling us your name, title and organization?

Adam Day

Adam Day, CEO, Clear Skies, Ltd.

Amanda French

Lovely. Tell us about Clear Skies Ltd. When was it founded? What does it do? What makes it special?

Adam Day

Clear Skies was founded in, let’s see, the first quarter of 2022, so not very long ago. The whole idea behind Clear Skies is to take data from multiple sources, bring it together, and use it to help research integrity. That’s what makes us special – our data is completely unique. But I mean ‘research integrity’ in a broad sense: not just research integrity as in dealing with misconduct or papermills, but the whole process. What can we do by bringing data together to help make research better or more robust?

Amanda French

I know that your flagship product is The Papermill Alarm, but does Clear Skies have other products that you’re working on?

Adam Day

We do. That being said, the Papermill Alarm has been a much bigger success than I had expected. Initially, the Papermill Alarm was one pipeline for detecting papermills, but then we came up with a bunch of other pipelines for that purpose. To avoid confusion, we rolled all of those services up into one product with simple, clear, output. That’s the Papermill Alarm.

Amanda French

When and where did you personally first hear about ROR?

Adam Day

I was at a FORCE11 conference in Edinburgh, and one of the sessions was about ROR. I wanted to do analysis at an institutional level. So, you want to say, “Okay, which institution is producing the most papers on some topic?” or “Which is the best-cited institution on some topic?” or maybe “Which institution in which country is growing the fastest?” Those are the sorts of questions that you might want to ask, and if you only have, say, addresses to go on, that’s a total nightmare. Completely impossible at industry-scale.

There are other systems for identifying institutions, but they are not easily connected with the data. When you look at a dataset like OpenAlex, a very large part of it has these ROR IDs, and you go, “Great, I could just use that" and suddenly all this institutional analysis is possible at any scale.

Amanda French

And then for what purpose did you begin using ROR?

Adam Day

At Clear Skies, I wanted to examine the papermill situation at an institutional level. Initially, this was part of sense-checking the results coming from the Papermill Alarm development process. ROR showed which institutions trigger the Papermill Alarm most often – and it turns out that the same institutions already have retractions relating to papermills. I doubt that’s a coincidence!

Amanda French

How important do you think that organization correlation is in detecting papermills? What are the other factors that are important in that?

Adam Day

There are a lot of factors in there. Institution is part of the picture. The key metrics that have proven the value of the Papermill Alarm are things like:

• Recall of known papermill-products. When we train the Papermill Alarm, we show it examples of papermill-products, but not all of them. We keep some aside. Then, after training, we show some non-papermill-products and those known papermill-products. If the Papermill Alarm can accurately flag the known papermill-products and flags none of the non-papermill-products, then we have good evidence that it works.

• Prediction of mass-retractions. There is a long and growing list of these. Given real-world data, the Papermill Alarm consistently identifies the potential for these events.

In practice, the fundamental thing is that there has to be a human-in-the-loop in the analysis of every case. It would be fantastic if we could just have this wonderful, magical machine that could detect all of the fraud and deal with it for us with 100% accuracy. But the reality is that you need to have humans in there who can exercise judgment over each case.

Imagine a plagiarism case, where you find two papers that are basically identical. And so you say to the author, “Hey, look, we found these two papers, your paper and this other one, and they’re the same. Did you copy this paper?” And the author might come back and say, “No, I didn’t. It just coincidentally happens that I wrote an identical paper to this other person who I have never met.” That is an excuse that comes up sometimes. And, of course, there has to be someone there who can look at the evidence, weigh it up carefully, and say “I don’t believe you.”

That’s it. Having institutional identifiers in there, it’s part of the puzzle. And it’s an important part of the puzzle. But there are limits to data analysis. Data analysis has got to help people to do the investigation – not try to do it for them.

Amanda French

What other persistent identifiers do you use in Papermill Alarm?

Adam Day

There are lots of things that come into the analysis. DOIs, frankly, are absolutely critical. Then there’s ORCID, which we use to some extent, although it is not universally adopted. There isn’t a perfect way to identify journals and publishers, but we get pretty far using a mix of OpenAlex’s IDs, ISSNs and Crossref’s member id.

Does the ROR identifier get added from the affiliation string?

Amanda French

Yes! We actually just published a case study with OpenAlex, which you may want to read. They did some pretty heavy lifting in terms of doing that string to ROR ID matching, too, and apparently it’s working quite well. The large majority of OpenAlex materials do have ROR IDs, and they are using exclusively ROR IDs for organizations right now.

Can I ask, do you do this inductively, where you look for interesting patterns in the data? Or do you start out with a hypothesis and then see whether or not it checks out based on your analysis?

Adam Day

The original Papermill Alarm started out with a hypothesis about what was going to be detectable and what wasn’t. And it turned out to be a lot more than I’d expected. But there have been plenty of times when I’ve been digging through data (something which I do more than I would care to admit). And then you’ll see something and think, “Oh, maybe I could look at this.” And it turns out that the thing you’re looking at is quite important. And also, more often than not, the thing you’re looking at turns out to not be important. So hypotheses come up, you know, as you’re digging through stuff, and you find things that way.

Amanda French

That sounds like good science, I’d say. Going back to something we were talking about earlier, how do you think that persistent identifiers like ROR can help with the overall project of trying to ensure research integrity?

Adam Day

On the one hand, it’s just about data integrity. ROR is a key part of the data. We need to connect these articles and these individuals to the institutions where they work. That needs to be done well. The same is true of all persistent identifiers – they allow you to connect up the data at scale. Getting persistent identifiers right means we have a complete and accurate picture of things.

If we’re talking about misconduct, then you might need to be able to contact the institution that the author is from. On an individual manuscript, it doesn’t matter if there’s no identifier – an address will do. But if you find some signal that is on manuscripts at scale, and you’ve got thousands of them, well, you need an identifier. You can’t go through them and try and search for every single one of those institutions.

But in terms of integrity generally, there’s also just this point about accountability. Who is accountable for the content of the paper? You’ve got the individual researchers, and you can identify them. But then who else is accountable? Well, the publisher is, because they are the ones who peer-reviewed it, and they’re putting their name on it. Then you’ve got the institution as well, and they’re also accountable. So when it comes to the difficult task of investigating misconduct, having the ROR institutional ID on there means you know who to call.

Amanda French

Do you do any work with funders at all?

Adam Day

We don’t. At least not yet.

Amanda French

I almost think of funders as more the regulatory or enforcement part. OpenAlex actually uses ROR to identify funders in its funder model as well, because funders are in scope for ROR, and we have lot of them. You may know that Crossref runs the Open Funder Registry, which has been curated by Elsevier, and we’ve just announced that the Open Funder Registry will eventually be deprecated in favor of ROR. ROR IDs to identify funders is the future.

Adam Day

Oh, great! I like that!

Amanda French

It may be a while. We’re thinking on the scale of about two years or so. But we’ve got more blog posts coming out about that.

Adam Day

I saw the announcement about Crossref and Retraction Watch. That’s great news.

Amanda French

Yes, it is! Glad you saw it and that you think so. Now, let me give you a chance to make some feature requests. What would make your life easier if ROR did something differently?

Adam Day

Being able to go down to the departmental level, that would certainly be useful in cases where you’re finding an institution where there are tons of problem papers going in, but it’s a gigantic institution and they’re putting out thousands of papers a month. When you’re finding dozens of problem papers coming out in all of these thousands, that might not actually tell us all that much. But then if we were to look at that at a department level, and see that those dozens are all coming from one department, that’s a really useful signal.

The last time I did some analysis on that, I think I wasn’t able to drill down. Maybe when some people have got their ROR ID, they’re going to have either the top-level one or one that’s lower down, not both.

Amanda French

All of those high-level relationships are in the ROR record, but in their implementations or in their analyses, not everybody will do that linking. Some do, though. To use OpenAlex as an example again, they relatively recently released in their API a thing they call institution lineage, where you can do a query on an organization to find any parent organizations it may have.

Adam Day

Oh that sounds very useful – I will look into that! Thanks!

Amanda French

What else do you want to tell us about ROR, or about your work, or about anything at all?

Adam Day

There is one thing I’d like to say, which is that the availability and openness of data like this is really important. Because if it is free and open and available, then it’s easy to work with. As soon as you get datasets where there are even little barriers to using them, that’s an issue. Something like authentication doesn’t really hold you back, but you can be using some tool that requires authentication, and then maybe the authentication breaks one day, and suddenly you don’t have your data anymore. Having nice, clean, big, open datasets like this is really valuable, especially for a problem like this one, a problem which does require taking a big step back and looking at all of the data together, and not having barriers in the way of getting parts of it.

That’s one of issues around, say, analysis of full-text data. You can get some of it, because it’s open access. There’s a dataset that Semantic Scholar pulled together and made available and things like that, but if you want all of it? No, that’s just not even remotely possible.

Amanda French

Even abstracts are difficult. Crossref has done a lot of work on open abstracts with its Initiative for Open Abstracts.

Adam Day

It’s a good thing to do, because we do use abstracts in papermill detection. Actually, one point about that, which is maybe a bit tangential, is that every now and then when someone finds a dodgy paper, they retract it, and they change the title to put the word “retracted” in. For me, if I’m trying to use that for training a machine learning model, that’s bad: I have to take that out. “Retracted” has to go. But that’s okay, I can do that. I can make a little rule to do that. But what happens sometimes is that they change the abstract. The abstract disappears, and then it gets replaced with something like “The publisher has removed this paper.” Now that’s not useful data anymore. It’s just gone. The publisher retracted this one, but we can’t help them stop the next one getting retracted. [For more on this topic, see the NISO Working Group on the Communication of Retractions, Removals, and Expressions of Concern. Comment is open on the working group’s best practices draft through December 2, 2023.]

That’s the thing I think is really important and valuable about ROR, is that being open and being available, it just makes so much stuff possible that wouldn’t be otherwise. It’s also one of the things to say to anyone wanting to get into stuff like this. If you want to do data science on published research, just go for it. You can use the ROR API, you can use the Crossref API, OpenAlex, all this stuff. They’re free. Go and do it. And that’s a great way for people to learn. I learned tons of stuff by just trying things like that. That freedom is one of the great things that we get from having resources like this.

Then there’s serendipitous discovery. If people are able to get into it and just mess around, they won’t just find what they are looking for. They’ll find other stuff as well. And who knows what that might be.

Amanda French

I’m glad you said that. Having the data be open is a really crucial part of what we think is important about ROR.

Adam Day

It’s like serendipitous discovery, right? If people are able to get into it and just mess around, eventually, you’ll get people just going, “Oh, look, you can do this.” And they would never have known if they didn’t have the freedom to play around with it, so it’s really important.

Amanda French

That’s great. Well, thank you for speaking with me.

Adam Day

Cool. Thanks. Bye.

Crossref and Elsevier have done the entire scholarly communication ecosystem a great service not just in creating and maintaining the Open Funder Registry (formerly FundRef), but also in promulgating its use by publishers and other systems. Thanks to the Open Funder Registry, it has never been easier to match funders to research outputs using open connection metadata, and the current moment is one in which linking funding to research outputs is of critical importance to ensure transparency and research integrity.

As more and more systems have begun using ROR for contributor affiliations, however, it has become clear that also using ROR for the all-important task of identifying funders provides many benefits:

• Streamlining processes by using one registry instead of two
• Reducing difficulty with ROR’s easy-to-use REST API, clear documentation, and web-based landing pages
• Being able to see and participate in ROR’s open, transparent metadata curation processes
• Supporting community-led and community-governed infrastructure

Current users of the Open Funder Registry can begin planning now to enjoy these benefits by switching to ROR for funder identification.

ROR has been working throughout 2022 and 2023 to ensure that ROR has excellent coverage of funders, and we intend to continue this work through 2024. Read on for a detailed account of the current overlap between the Open Funder Registry and ROR and to learn more about the ROR / Funder Registry Overlap tool we’ve built to help you conduct your own analyses.

Comparing at the record level

Much of the information included in ROR and Open Funder Registry records is similar. Both ROR records and Open Funder Registry records include a globally unique identifier for the organization, a name for the organization, alternate names and acronyms for the organization, the location of the organization, the organization’s type, and relationships to other organizations. If we examine the record for UK Research and Innovation (UKRI) in both the Open Funder Registry and ROR, for instance, we can see that the associated metadata is similar.

Note that the ROR record includes a reference to the corresponding Funder ID, and note that both the ROR record and the Open Funder Registry record list the same nine “child” organizations for UKRI (e.g., Research England). Each of those subsidiary organizations has its own identifier in both registries.

One difference between the metadata in the Open Funder Registry and the metadata in ROR concerns organization type. The Open Funder Registry gives each funder a type of government or private with associated subtypes for each. Government funder subtypes include federal and non-federal, while private funder subtypes include academic, corporate, foundation, international, professional associations and societies, and other non-profit. ROR also supports organization types and currently allows eight options: Archive, Company, Education, Facility, Government, Healthcare, Nonprofit, and Other.

In version 2.0 of the ROR metadata schema, dataset, and API, which is due to be released early in 2024, ROR will add support for a “funder” organization type and will add this type to all ROR records that include a Funder ID as an addition to the existing organization type value. This means that early in 2024, the version 2.0 ROR record for UKRI will include organization types of both government and funder. In the meantime, as we explain in our documentation of the ROR API advanced query parameter, the list of all active ROR records that currently include a Funder ID can be retrieved using the following query:

https://api.ror.org/organizations?query.advanced=_exists_:external_ids.FundRef.all

Version 2.0 of the ROR schema and API is open for beta testing through October 16th, 2023 – current users of the Open Funder Registry are more than welcome to participate and to give us their feedback.

At the record level, then, ROR and the Open Funder Registry are far more similar than they are different. Let’s look now at the general overlap between the two registries to learn how well-suited ROR data is for the purpose of identifying funders.

Comparing at the registry level

Most of the Funder Registry records that see the most use already have equivalent records in ROR, which is good news for users of the Funder Registry. Throughout 2022 and into 2023, the ROR team has been working diligently to reconcile the Open Funder Registry with ROR, and this reconciliation work has resulted in the addition of 631 new ROR records identifying funders. Moreover, we have modified 6,080 ROR records to add Funder ID mappings and to ensure that these records include key metadata from the Open Funder Registry.

The project to reconcile ROR with the Open Funder Registry has been focused on Funder IDs that have existing uses, or “assertions,” in Crossref records, and the work has been prioritized on the basis of the total number of assertions. The Funder Registry includes identifiers for about 35,000 funding organizations, and of these, nearly 33,000 are used at least once in Crossref metadata. More than 8 million content items registered with Crossref include Funder IDs,² and since many of those items cite more than one funder, there are over 13 million total funding assertions in Crossref metadata.³

To analyze the overlap between ROR and the Funder Registry, we have built and made publicly available the ROR / Funder Registry Overlap tool. This tool shows that after ROR’s reconciliation efforts to date, 94.1% of the existing Funder ID assertions in Crossref metadata can be matched to ROR IDs. We expect this percentage to increase as the reconciliation work continues.

The pie chart on the left shows that of the 35,000+ funders in the Funder Registry, more than 19,000, or a small majority of 54.6%, are represented in ROR, usually on a one-to-one basis. However, about 2,000 Funder IDs in the Funder Registry are never used in Crossref metadata, and many of the remaining 33,000 are used only once. Individual funders who issue many awards can account for a high number of funding assertions, and ROR has therefore focused its Open Funder Registry reconciliation work on these large funders. For example, the United States National Science Foundation currently accounts for over 332,000 funding assertions by itself, and with its many associated laboratories and agencies, it accounts for over 412,000 funding assertions in works registered with Crossref.⁴ The pie chart on the right shows that of the more than 13 million uses of Funder IDs in Crossref metadata, nearly 12.5 million have a corresponding ROR ID.

The total overlap is even higher for DataCite, with 96.7% of funding assertions in DataCite having corresponding ROR IDs, as shown below.

Just as in the previous view, the pie chart on the left shows that a majority of Funder IDs have an equivalent ROR ID, while the pie chart on the right shows that the overwhelming majority of DataCite funding assertions can be matched to a ROR ID, with available ROR IDs for 941,049 of the 973,072 funding assertions in DataCite records.

In addition, while the preceding chart shows the number of funder assertions in DataCite records that could be matched to a ROR ID, the data on the actual number of funder assertions in DataCite records that currently use a ROR ID is also impressive. About 1.5 million DataCite records cite a funder in the fundingReferences element, and over 1.3 million of those also include connection metadata in the form of an identifier for that funder. As of this writing, the largest proportion of those 1.3 million funder identifiers are Funder IDs at 883,098 uses. ROR, however, is the next most commonly used identifier at 357,568 uses.

Clearly, then, ROR is ready to take on the work that the Open Funder Registry is currently doing with regard to funding data in Crossref and DataCite records.

ROR’s curation team plans to continue adding entities listed in the Open Funder Registry to ROR based on priority and usage. You can help us determine what needs to be done by analyzing your own funder data with the ROR / Funder Registry Overlap tool that we’ve made available to help you in this transition.

Analyzing your own funder data

If you are a funder or if you work with Funder IDs, you can use the ROR / Funder Registry Overlap tool to look up any funder in the Open Funder Registry and find its equivalent ROR record. If the information in the ROR record needs a correction, or if there is no matching ROR record, you can request that ROR make the change from within the tool.

You can also look up how many of a particular Crossref member’s funding assertions can be mapped to ROR IDs by choosing “Crossref - Overlap by Member” from the views on the left side of the screen.

For instance, Crossref member the Royal Society of Chemistry (RSC) has as of this writing used Funder IDs over 478,000 times in Crossref metadata, and nearly 93% of these assertions can be matched to a ROR ID. Of the 9,000+ funders that RSC publications currently acknowledge in Crossref metadata, nearly 73% have corresponding ROR IDs.

Remember, too, that the project of reconciling ROR to the Funder Registry has to date focused on making sure that frequently-asserted funders are accurately and fully represented in ROR rather than on trying to create a new ROR record for every single Funder Registry record. The work to reconcile the Funder Registry to ROR is ongoing, and over time we fully expect even more Funder IDs to have matching ROR IDs.

We hope you find the ROR / Funder Registry Overlap tool useful in assessing funding data. We have made the source code available on GitHub for those who wish to examine or clone it.

Get ready and get in touch

If you’re already a user of the Open Funder Registry, you should start planning now to switch to ROR – we’re happy to work with you to make your transition to ROR easier. If you haven’t yet adapted your systems to capture funding data in a structured way that uses persistent identifiers, we’re also happy to work with you to help you integrate ROR. We all need clean, consistent, open, and interoperable information about all kinds of research organizations, including funders.

In the coming months, Crossref will be publishing at least one blog post detailing the technical work they are doing to build comprehensive support for ROR into Crossref’s services, and Crossref and ROR will continue to consult with key users of the Open Funder Registry to learn what they want and need.

Meanwhile, if you have questions or comments about ROR and the Open Funder Registry or if you’d like to contribute to our ongoing work to improve information about funders in ROR, get in touch! You can book a meeting with ROR Technical Community Manager Amanda French or write to her at support@ror.org. We look forward to hearing from you as all we prepare for these exciting changes together.

Throughout 2022 and into 2023, ROR gathered community feedback about how to handle schema and API versioning and about what should be included in a new version of ROR’s metadata schema. Today, we are thrilled to release a beta version of ROR’s first major schema and API update. Version 2 of the ROR API and metadata schema is open to the public for testing through October 16, 2023. As a community-driven initiative, ROR relies on input from its stakeholders to ensure that new and updated features serve the needs of its wide array of users and use cases.

We plan to incorporate changes from this beta test this fall and to make API v2 available in production in early 2024. The current version, version 1, will remain the default version until at least early 2025.

A full account of the changes and detailed instructions for participating in the beta test are available in our v2 beta test documentation, but we provide some highlights below.

What’s new in v2 of the ROR metadata schema

• Name information previously in name, acronyms, aliases, and labels fields is now contained in a single parent field names with subfields lang, value and types.

• Location information previously in addresses field is now in locations field with subfields geonames_id and geoneames_details. Many fields containing very granular information derived from Geonames have been removed, as this information is available directly from Geonames.

• Website/domain information previously in links and wikipedia_url have been combined into a single parent field links with subfields type and value. The ip_addresses field has been removed, and a new domains field has been added.

• External identifiers information has been restructured within the existing external_ids field. Each item in external_ids now has subfields type, all and preferred.

• Administrative information was not included previously. A new parent field admin has been added, which contains subfields created and last_modified. Each of those subfields contains additional subfields date and schema_version.

• Controlled lists previously had variations in casing. For example, values in the types and relationships.type fields began with an uppercase character, while values in status were lowercase and external ID types contained a variety of casings. In v2, allowed values in controlled lists are consistently lowercase, with the exception of country codes derived from ISO-8166, which are uppercase per the standard.

What’s new in v2 of the ROR API

Version 2 of the ROR REST API includes all the same search and retrieval functionality as the current version (v1) of the ROR REST API, but with ROR records in responses formatted according to the v2 schema. For all ROR API requests, the API version is specified in the path portion of the request, as in the following example:

curl https://api.dev.ror.org/v2/organizations

Sample v2 ROR data

A sample data dump containing both v1 and v2 files is available in the locations below:

• Zenodo sandbox: https://sandbox.zenodo.org/record/1241263
• GitHub: https://github.com/ror-community/ror-data-test/blob/main/v1.32-2023-09-14-ror-data.zip

These files are for review and testing purposes only and are not official ROR release files. Note too that there are several new fields/subfields in v2, and the dataset used in the beta has not been fully updated with values in all new fields/subfields. In particular:

• Created/last modified dates HAVE been added to all records in the new admin field, using actual dates from GRID and ROR data releases.
• Domains HAVE NOT been added. The new domains field is currently an empty list for all records.
• Language codes for items in the new names field are only included for names inherited from the labels field in the current schema. Language codes HAVE NOT been added for names inherited from the name and aliases fields in the current schema.

We plan to add data to empty fields in v2 over the coming months, carefully curating the records to ensure accuracy.

How can I provide feedback on the v2 beta?

1. Read the complete beta documentation and testing instructions.

2. Check the ROR schema & API v2 beta project board to see if the same or a closely related issue has already been reported. If so, please add comments to that issue.

3. If you have new feedback that’s not related to another issue on the ROR schema & API v2 beta project board, submit a GitHub issue using the ROR schema & API v2 beta feedback template.

All issues will be reviewed by ROR staff, and we may contact you using your GitHub handle to request additional information. If you are not a GitHub user or prefer to submit your feedback privately, please email support@ror.org . Please provide your feedback by October 16, 2023.

We are grateful for your help! Following the close of the beta test, we will review all reports and work to incorporate the improvements before we release v2 into production.

Thanks, as always, to everyone who works to make ROR the best it can be.

Amanda French

Thanks so much for talking to Adam and me, Justin. Can you tell us your name, title and organization?

Justin Barrett

My name is Justin Barrett. I am the lead Machine Learning Engineer for OpenAlex, which is a part of the OurResearch group. OpenAlex has been around since 2021, and I joined pretty early on, so I’ve been working for them for a little under two years now.

Amanda French

Tell us about OurResearch, and tell us about OpenAlex.

Justin Barrett

The overall goal of OurResearch is to increase open access, open data, and open science for everyone. OurResearch has several different projects to promote that. There’s Unpaywall, which tries to get Open Access articles to end users. There’s Impact Story, which I believe is the first project that our CEO Jason Priem created with Heather Piwowar, who used to work at OurResearch and was a co-founder. There are a lot of projects, and all of them try to make open science a reality for more and more people. And that’s basically what OpenAlex is continuing to do. We have established a large database of research that is completely open for everyone. If they want to, they can download a snapshot of our entire database and use that any way that they want to – to create recommendations, to create their own website, to create their own analytics. It’s really just empowering people to do that.

Amanda French

Great. And can you tell us a little bit more about the origin of OpenAlex?

Justin Barrett

Microsoft built and maintained the Microsoft Academic Graph, MAG, and then decided to sunset that whole system. And at that point, Jason and others in OurResearch decided that it was a good time to try and take over what MAG was doing, because they were serving a pretty important function for a lot of people. They knew that there were a lot of people that were going to be missing out with MAG not updating their database anymore. OpenAlex was created to jump on that opportunity in order to continue to provide researchers, academics, with access to the data in MAG coming in through Crossref, coming in through PubMed, coming in through a lot of different data sources. OpenAlex is just trying to fill that gap.

Amanda French

And were you party to any of the discussions with Microsoft about that?

Justin Barrett

I was not. I came on after the decision was made to carry on MAG’s work. Jason contacted me because there were models that MAG already had within their system that Jason needed to have replicated at the very least and then eventually improved upon. The first project that I worked on was creating a concept tagger for papers, a topic tagger based on the title of the paper and on the abstract if we had it. That project was the first where we were trying to bridge the gap between just getting the data in and enriching it with some of the features that MAG was enriching all of their data with.

Concepts was where it started, and institution parsing was next. There have since been a couple of different iterations for both of those models. Then it was author disambiguation, and that’s actually the current project that I’m working on, to completely revamp that system so that it’s improved. We’re hoping to release that this week or next week. [Read the release announcement for OpenAlex Authors.]

Amanda French

Can I ask you about your background and why Jason came to you when he needed somebody to work on these models?

Justin Barrett

That’s a good question. I guess it comes down to just luck. He hired me through Upwork, which is a pretty popular platform for hiring contractors for short-term projects. I went to school for aerospace engineering, and I got my master’s in mechanical engineering. I have a lot of coding experience from my years in engineering, and during my master’s I got into Python a lot more. When I went through my master’s, I figured out that I didn’t want to do mechanical engineering anymore, and I did a lot of self-learning through Coursera and Udemy and basically made that jump over to data science.

I pretty quickly started specializing in deep learning applications. Natural language processing, NLP, and deep learning are what I like to do and what I’m pretty competent in. Jason found me when I was about a couple of years into Upwork. I also had a job at Advance Auto Parts as a data scientist, so I had been doing a lot of AI and machine learning work. And I guess when I applied for the job on Upwork, Jason liked my experience. Then we met, and it all went from there.

Amanda French

When and where did you personally first hear about ROR?

Justin Barrett

When I was creating my first institution parsing model here at OpenAlex. That was my first time hearing about it, and so I wasn’t aware of how much data was available in ROR. My CEO, Jason Priem, told me that ROR would be a good source of information for institutions.

At that time, for our first model, we inherited all of our institutions from MAG, and so we had non-ROR institutions included in there. I wasn’t as eager to go to ROR completely, because I had enough institutions where I thought, “All right, I need to not depend on ROR, because a lot these institutions are not included in ROR.” And so I used ROR for gathering additional information about the ROR institutions, like a city and a state – basically, I used it to supplement all the data that I couldn’t find within MAG. City, state, and country were the main things that I used for the first version.

When we started to make the second version of the institution parsing model, I got a lot deeper into ROR, and that’s when I started to use a lot more of the data, like the aliases and the acronyms and the relationships. ROR became a lot more central to our model, because I was using it so much, and then also because we made the decision to not include non-ROR institutions in our database. We switched to completely ROR institutions. That is something that we could possibly change in the future, if we maybe generate a list of institutions that we want to also include in our database, but we made that decision to kind of streamline all of our data, because it’s a lot easier to keep track. But that’s basically how I started hearing about ROR and started using that data.

Amanda French

Were there institutional identifiers in MAG that you inherited?

Justin Barrett

MAG had an institution identifier that was an “I” followed by whatever number MAG had assigned to that institution, so those were the IDs that I used. We had a mapping from those institution IDs to a ROR ID if there was one available. Those were the institution IDs that I used at first, and now that we’re using ROR only, there’s basically a one-to-one mapping of ROR IDs to OpenAlex’s Institution IDs.

Adam Buttrick

Can you talk a little bit more about this streamlining? Why did you pivot to narrowing the OpenAlex institution model to use only ROR IDs?

Justin Barrett

We did a little bit of analysis at first to see “Okay, what’s the distribution in OpenAlex as far as the number of papers for ROR institutions versus non-ROR institutions?” Through that analysis, we found that about 97% of institution assignments were ROR institutions.

In addition to that, a lot of our non-ROR institutions were just garbage. They weren’t linking to any Wikidata page, they didn’t have any information about location, and some of them were duplicates of ROR entities. The more we looked into it, the more we wanted to just start with ROR, and then maybe in the future, we could start adding institutions on top of ROR.

So we just made that decision, and it made my job a lot easier when it came to assigning institutions. It cleaned up the training data, it allowed me to try out different things with the training data, and it just made things a lot easier.

Amanda French

Have we ever talked with you about adding OpenAlex institutions to ROR that don’t currently have a ROR ID?

Adam Buttrick

I don’t know if we talked to Justin about it, but I have done it, because I used to look at OpenAlex a lot and say “What isn’t in here? What looks valid?” My conclusion in doing that was very similar to Justin’s, that there’s a bunch of things that are essentially duplicates of existing organizations, and there were some defunct organizations, like organizations from the Soviet Union that don’t exist anymore, that kind of stuff.

Justin Barrett

There was also a separate process that happened when we converted to ROR-only where we looked at some of those top institutions that were considered non-ROR, the ones that had the most papers associated with them. We made sure that they were not supposed to link to an actual ROR ID.

We ended up merging a lot of our institution IDs to make sure that we weren’t dropping too many, and we found a large percentage of them that should have linked to a ROR and didn’t or that had a separate ROR for that entity. That was another member of our team who took care of that process. At the end of the day, we currently only have ROR institutions that are getting tagged for our affiliation strings.

Adam Buttrick

Just to give the people who will be reading this interview some perspective, what’s the scale of works in OpenAlex that are being indexed with ROR IDs?

Justin Barrett

I think OpenAlex has around 243 million works as of right now. Of course, not all of those have institution strings or affiliation strings that come with them. The last time I checked, about 80% of the new data we have coming in has an affiliation string. When you go back to some of the old data in MAG, papers from the 1960s and 1970s, a lot of those are missing institutional strings, but for our more current data, I would say about 80% at least have an affiliation string with them, so they are getting matched to a ROR ID.

Adam Buttrick

That’s great. But it also kind of highlights the need for publisher partners to contribute those, to impose that little bit of order onto the chaos of it and not let the metadata wither in the same way that the historic data has done.

Justin Barrett

Yeah, it would be nice if it came in with a ROR ID from the publisher.

Adam Buttrick

We’re working on that!

Amanda French

I’d like to hear a bit more about how the matching that you do works and how you get new ROR data.

Justin Barrett

We get the ROR data a couple of different ways. When I was creating this model, I used the most recent data dump to get an updated snapshot of all the ROR data, and then that’s what I used to perform all the string matching. Currently, that snapshot is what the model is using in order to match those strings.

In the future, we are going to be doing that slightly differently, because we will be implementing some processes to use the ROR API. I believe we’re using the ROR API to update some institution data in our database with ROR data, but that is not propagating through our whole system as of right now. I didn’t work on that project, that was one of our data engineers. But I believe at some point in the future, we will ping the ROR API once a day to check to see if any of the data has changed for specific institutions, and if there’s a change, we will take that data and put it into our system.

When I get done with the author disambiguation project, institutions are something that I’m about to start looking at again, and because we are now populating our own database with ROR data, that gives me the opportunity to constantly update the strings that it’s matching on. The process of matching those strings will be a little more streamlined to current data, as opposed to right now where it’s just a snapshot from the past, which probably works most of the time, but which doesn’t give me new data that you guys are providing, and which doesn’t correct any data that might be wrong from that snapshot. I’m looking forward to getting back into institutions and making it more of a sustainable pipeline as opposed to what it is right now, where it’s kind of a point in time model.

Adam Buttrick

I have a related question. Did you work on the new funder model that has been integrated? I saw that the OpenAlex funder model also integrated ROR, so I wondered if the ROR data was being pulled in in the same way.

Justin Barrett

I did not work on that, so I could not speak to the funders. But I know that was a big project that we were pretty excited about releasing.

Adam Buttrick

It was very forward thinking to include ROR ID mappings, because over the long term, the Funder Registry and ROR are intended to merge. It’s a long ways out, but you’ll be ahead of the game, because you already have a Funder ID to ROR ID mapping in the Funder object, so you can just privilege one ID or the other.

Justin Barrett

I’m sure it won’t be that easy at the time, but we’ll have a head start. I wish it was always as easy as a one-to-one mapping from one thing to another.

We have the alpha version of our user interface up right now, which is very rough, but it gives you kind of a preview of what we will probably have within the next month or two. It’ll be really nice: you’ll be able to look and see funders and institutions and all that information pretty easily. Before, you could look at that information, but it wasn’t really user friendly, so we’re pretty excited about the UI that’s coming.

Adam Buttrick

Do you know if you’ll leverage things like ROR hierarchies and relationships in the UI such that the works of child organizations can be associated with parent organizations? Some people have done that, others have not, so I wondered what OpenAlex’s approach would be.

Justin Barrett

At first, no. At first, it’ll probably just be a mapping to each institution’s own ROR ID in the alpha version of the UI. Once we start getting into subsequent versions, I think that we’ll start looking to link all of the data a little bit better, so yes, at that point we’d probably add ROR institutions with the parents and children and all those relationships. However, we have recently started leveraging some of the ROR relationships and hierarchies in our API. It is a separate feature called ‘lineage’ which links the children of an institution with the parent only if a parent was tagged using our institution parser. Eventually this feature will make it into our UI but for now it is only accessible through our API. Concepts also have a defined hierarchy that we would be most likely looking to leverage at some point in the future. So yes, there are a couple of things that we’ll be looking to improve in our UI, but getting a better version out there to start is our top goal.

Adam Buttrick

I think everyone’s really excited for it. I know from the OpenAlex user group that people are super engaged with everything you’re doing with the UI.

Amanda French

You’ve already touched on this, but what are some of the major things on the OpenAlex roadmap that are coming up?

Justin Barrett

As far as our roadmap, author disambiguation is at the top of my list, and it has been for the last four or five months. Originally, we had a very patchwork model that we built when we first came away from MAG just so that we wouldn’t be minting new author IDs for every author that came into our system. We needed to improve on that, which we did with the v2 model, which is what we have now. But there were still a lot of improvements that we could have made.

This version 3 model should improve our author disambiguation system a lot. Right now we get a lot of feedback like, “Hey, my name is split into 20 different profiles, and my ORCID isn’t in my top profile, can we fix this?” And we say, “Yes, we would love to fix that.” I would say that the system we currently have isn’t good with merging clusters or taking them away. We have to do it very manually, which is okay at the moment, because there aren’t a ton of users who are asking us for this. We’re getting a lower number of requests than we can deal with. But eventually, if we want to scale up, and we have thousands of users or hundreds of thousands of users asking us to change their author profiles, we want to make sure that the system we’re building now can handle that.

And then we also want to make sure that we have a system in place for them to make those changes themselves. One of the big things that’s going to be coming out eventually is that we’re going to let authors request changes to their own profile. If they see that a work assigned to them doesn’t belong to them, they’ll be able to submit a request to get that work removed, and then our system will either try to assign it to the next most likely author or will create a new author for that work.

We’ll also allow authors to merge two profiles, and we’ll allow them to add works to their profile, and we’ll have some process for authors to claim a profile on OpenAlex. It should be a pretty cool improvement, but to do it we need a good author disambiguation model and good author clusters to begin with. That’s what our current goal is.

We have a lot of different projects going on. One of our team members is working on increasing the number of statistics, the “science of science” metrics that we have within our system to make filtering easier through the API. We want to give people more ways to filter our data for metrics like “number of works within the last five years” or “percentage of citations within the last five years.” Right now there just aren’t as many metrics to filter on.

We also have general system architecture projects. We’re trying to make it so that our data is updated much more quickly so that when we eventually get to a place where users are sending us requests for data to be changed, those results will propagate through our system much more quickly than they do right now. One of our engineers has been working on that for probably the past week or two, so I’d say that’s another big project.

Amanda French

Do you have any sense of who the users of the OpenAlex API are, roughly?

Justin Barrett

We have some idea because of the requests that we’re getting – the problem reports, or bug reports. We have a group of people that submit tickets for any number of reasons, and there are definitely a lot of graduate students that are asking us, “Hey, can you increase the limits for the API for this email address so that I can pull up more data?” Just based on the number of grad students that message us about increasing their API limits, I would think that those are one of our largest groups of users for the API. I can’t imagine many other groups that are more prominent for API usage.

Amanda French

What are they doing with the OpenAlex API? What are they using it for?

Justin Barrett

That is a good question. When people submit support tickets for us, a lot of the time they include the reason why they need the API increase. We don’t need that reason; all we need is an email from you requesting more data from the API, and usually we just give the user a higher limit. But a lot of people will tell us about the research they’re doing. They want to analyze trends. Some of them want to analyze trends because they want to figure out where they should research, what topic they should explore, where they should submit a paper.

There are a lot of interesting support tickets that come in that tell us why people are using our API, but we don’t really analyze API usage as far as what people are searching for in the API.

Adam Buttrick

Could you tell us a little bit more about the author disambiguation that you’re doing? Your new author model is more tightly coupled with ORCID, is that correct?

Justin Barrett

Yes. For our new model, we are treating ORCID as the source of truth for author data. That comes with some drawbacks, because ORCID data isn’t always reliable, so there are issues with that as well. But our hope is that over time those issues will get resolved, and our data will be resolved with it.

We have a lot of ORCID data in our system. We realized that we had gaps in our data where Crossref wasn’t giving us an ORCID, possibly. Something was happening where you’d look in the ORCID public data dump, and there’d be a DOI that had an ORCID attached, and we’d look in our system and we would not have that ORCID. We’ve reconciled a lot of those differences.

ORCID is the basis of how our clustering starts. The ORCID clusters are the clusters we start with, and then we have some algorithms that are matching works to those clusters. If they don’t work, they don’t match with those ORCID clusters, then they are creating new clusters.

So yes, ORCID is at the center of what we do, at least for author disambiguation. It was able to give us a good training dataset. That’s the hardest thing about author disambiguation, is finding good training data. Which brings us right back to ROR, because that was also giving us good training data for training our model, because sometimes we couldn’t depend on the MAG data to give us everything we needed. It’s been very helpful.

Adam Buttrick

I think that OpenAlex is going to become kind of a more interesting source of truth, or maybe a source for building services on top of. I don’t know if you’ve heard that ROR is now the primary institutional identifier behind ORCID, because Ringgold has opted not to renew their agreement with ORCID around data sharing for organizational data. And because of the association between ROR IDs and authors’ works, and now ORCID IDs by way of your new author model, I think that as a result of that people can make a lot more connections that probably weren’t even exposed in either the Crossref API or the ORCID API. It’s very cool.

Justin Barrett

ROR is playing a huge part in the author disambiguation as well, because we’re using institutions to determine if authors are the same. Unfortunately, you can’t only use institutions to decide that. You can’t only use co-authors, you can’t only use topics, so we’re using a mixture of all of those things. We’re getting that institution from the author’s affiliation string and parsing it, and that’s going directly into our author disambiguation model to try and match that institution. So ROR, again, playing a vital part in our system.

Amanda French

Awesome. Does the OpenAlex data model support multiple affiliations for authors? I would assume it does.

Justin Barrett

Yes. We did not used to. Our first version of the institution parser was pretty good, but it could not handle strings that had multiple institutions in it. And so we worked a lot with CWTS. They help create the Leiden ranking.

They took samples of our data, and they came back with suggestions for ways that we could improve it. We ended up incorporating a lot of those suggestions into our v2 model, and one of the main ones was to fix the problem that we could not handle multiple institutions in a single string.

The v2 model that we created was highly dependent on ROR. The models in the first version of the institution parser used some ROR data, but they didn’t really integrate ROR into our system. For v2, the way that we figured out to get multiple institutions in a string is to take the top 20 most likely institutions that the model says might match and then to use some custom string matching on those top 20 in order to see which ones would match. That has been very successful.

There are issues when it comes to longer strings that have more than three or four institutions in them. As you can imagine, the more institutions contained within a single string, the harder it is for the model to point you towards all of them accurately. There’s definitely degradation when there’s that many institutions within a string, but if you’re looking at two or three institutions in a string, sometimes four or five, it can normally get all of them. It’s pretty good.

Amanda French

Cool. Now I want to give you a chance to make some feature requests for ROR. What should we do that would make your life easier?

Justin Barrett

I know that the relationships between ROR institutions are very tricky sometimes. I see a lot of weird relationships, and I don’t really understand how they work, and maybe that is actually how they are. I think that the next step for us is to use that relationship data. I would hope that when I start using it, it’s ready to go, but I don’t know if that’ll be the case. We are working right now to look at French institutions. I don’t know if you are familiar with how complicated that is.

Adam Buttrick

I’m working on a project with the Ministry of Higher Education and Research in France, and it can be challenging to reconcile local conceptions of relationship hierarchies with ROR’s data, as you say, and also what’s an organization and what’s not an organization. We’re doing a lot of work on that. [Release v1.30 of the ROR Registry includes a great many updates to French institutions as the result of this intensive curation work.]

Justin Barrett

It was something that I saw when I was training the model. You see “CNRS” everywhere, and you see a bunch of different acronyms. I just assumed, “Okay, I see that acronym, and I know that the words in that acronym are this, and so, sure, we match that.” But in reality, it’s not that simple. And of course, we do not have those relationships correct with our current institution model. And so that’s something that we are definitely about to tackle.

It’s one of the reasons I’m going to be looking into institutions again is to try and figure out how to improve the French part of our affiliation matching, even though I know that it’s very confusing. There are people that have solutions to that. There’s a company, SIRIS Academic, that has been developing some models to specifically deal with French institutions. We might be relying more on the open source community to come up with a solution to that, because I don’t speak French, and I couldn’t tell you the ins and outs of the French academic system or the research organizations, so it’ll be helpful to get other people’s points of view.

Adam Buttrick

You should keep us in the loop about that work. To give you a perspective on what we’re doing on the curation side, the Ministry of Higher Education and Research in France maintains their own registry of research organizations and research structures.

Justin Barrett

Is that HAL?

Adam Buttrick

No, it’s separate from HAL. It’s called RNSR, the Répertoire national des structures de recherche. It enumerates all the French research structures which exist at various levels of solidity.

We’re working on a project to include various French research units, as well as their relationships to their parent organizations. We’re starting out by improving our coverage for what are called UMRs, Unités mixte de recherche, or “mixed research units.” We’re adding a bunch of them, making sure that they’re better disambiguated, and that they have the correct parents, based on the RNSR data.

In general, we’re doing a lot of work around improving French research organizations, and are being graciously supported by the French government to do that through the French Committee for Open Science. Hopefully, that will accrue benefit to OpenAlex. But I’d also be interested to know what you find as a result of doing the analysis, because I’m sure there are things that we’ve missed.

Justin Barrett

You will hear something probably within the next month or two. I think that one of the other things is that there are sometimes universities that have many institutes underneath them, and I think sometimes not all of those institutes are in ROR, and not all of those institutes are linked to the correct universities. That’s another thing that I have seen that could be improved. I mean, the ROR data’s very useful to me, so I can’t really think of anything that I would need to have it improved. It has been very useful to me so far.

Amanda French

Great. Good to know.

Adam Buttrick

It is really interesting to hear from someone who else is dealing with it from an aggregate perspective, though, because we’re so used to talking to integrators who are dealing with it on a more one to one basis, or a more regional basis.

Justin Barrett

Yes, I’m dealing with them all.

Amanda French

What else would you like to say about ROR, or about version three of the institution matching that you’re planning, or about anything at all?

Justin Barrett

I can say that one of the things that was most useful for us was that with all the information you can get in the ROR data, we were able to artificially create affiliation strings. We get raw affiliation strings from a bunch of different sources, but not all of the ROR institutions have affiliations in our system, and not all of them have a lot of affiliation. Some institutions have three or four affiliated works, while other institutions have thousands.

We built a process for creating artificial affiliation strings using certain patterns that you normally see in an affiliation string, and that was able to increase our coverage of ROR institutions. Maybe we had those strings in our system, but we weren’t connecting them to a ROR institution because MAG never connected to them. Or maybe there’s an institution that one day will give us papers that we need to parse, but they haven’t given us any yet. This process allows us to get rid of a cold start problem where you don’t have any data about an institution, so when it comes into your system it does not get tagged because our model has never seen it before.

Beginning at 03:34, watch Justin’s presentation at the 2023 ROR Annual Meeting about OpenAlex’s institutional parsing and how they used ROR to generate artificial author affiliation strings to use as training data.

The artificial strings have made it a lot more robust of a model, and we wouldn’t have been able to do that without the ROR data, because we would have just had to use existing affiliation strings. If we wanted to add more data, we would have had to find each individual ROR institution that didn’t have any data in our system and do manual string searching in order to get all of that training data, which would be very time-consuming. That’s definitely something that we could potentially do one day, but we were looking for something that was a lot quicker, because the timeline was pretty short for the v2 institution parsing model.

So yes: as I said, for v3, I think I’ll grab an updated snapshot of the ROR data so that I’m working with the latest and greatest, and I’ll probably look into using our database, because we’re grabbing data from ROR daily. I’ll probably look into having those strings added to a dictionary that I keep up to with strings to match to ROR. It’ll be fun to continue to develop the institutions model, because it’s already pretty good at what it does, so just finding areas where it’s not doing well and probably using ROR data to make it better will be a fun little problem.

Adam Buttrick

I just want to say that your fake affiliation thing was a really brilliant piece of work. When I was reading through your code and I saw that, I thought, “Oh, this is genius.”

Justin Barrett

I truly can’t take all the credit for it. I was in a meeting with Jason Priem, and he mentioned something about using the ROR data. He asked if we could use the ROR data to train on, and I said no, because we had non-ROR institutions we were looking to match. But then I realized we could do it, because affiliation strings have a standard syntax. It was probably a joint idea – it wasn’t just me. But it is a very good idea. Artificial data creation for non-structured data is almost impossible, but we’re dealing with semi-structured data, which lends itself better to artificially creating something. So yes, that worked out pretty well.

Adam Buttrick

I did actually write a script to recreate what Justin did, to take the ROR data dump and convert it into a set of fake affiliation strings. I don’t know if it exactly follows the patterns that OpenAlex used, but others can use it to do the same thing, if they want. If they’re building their own models, they want to supplement, they want to train, they can just pull the ROR data, run it through the script and do that.

I also want to say that it’s fantastic that OpenAlex does all this work open source so that other people can see it and read it, because I was able to just immediately go through and read all of the code for the institution model, see how it was working, understand it, and read the white paper. And that’s really fantastic, how much is being given back.

As we’re looking at improving our affiliation matching service, we have that benchmark, we have that thought process, so we don’t have a cold start problem for improving our service, because it’s all just out there in the open. There are so many other matching services for institutional identifiers that are closed and proprietary, and you don’t have any idea how they’re doing any of the matching or its level of correctness or its scores or anything like that. So what you’re doing, that’s great.

Justin Barrett

A lot of people aren’t used to how open we are with our data. People have asked, “Is there any way you could show me the code for this?” and I’ve said, “Oh, it’s all in this repository right here. And here’s a paper that explains how to do it. And here’s the artifacts, if you want to grab these artifacts, you can just download them to your account.” That’s the one thing that Jason has said from day one. We are completely open. We want everyone to have our data. We want everyone to have everything so that there’s no question of us being an open data, open science company. That is that’s our number one goal. That’s been the number one goal since OurResearch, and OpenAlex will continue to have that goal until we exist no more. But hopefully that will be not for a long time.

Adam Buttrick

It’s great. And I think it accrues benefit to ROR, just as much as ROR accrues benefit to OpenAlex, that exchange of the open code, the open data, everything. Speaking of which, as part of our affiliation matching improvement, we put new training datasets in a repository. It’s a large set of affiliation strings that were manually paired to ROR IDs at the American Physical Society. They’re all manually assigned, so it’s good data, and it’s about 15,000 pairs.

Justin Barrett

Oh wow. That’s a lot.

Adam Buttrick

It’s good for test scores. We also pulled all of the publisher-submitted items out of the Crossref API, which you can also use as a good benchmark. Now, of course, we don’t know necessarily how those were assigned, because a publisher could use some matching service that we don’t know about to assign their ROR IDs, but that’s another good one. But the APS data in particular is very good.

Justin Barrett

That is good, because there aren’t many good benchmarks, especially ones that match directly to ROR and go straight from a raw string, so it’s good to get a lot of those. I know that the French company that we’re working with right now, SIRIS Academic, they have also created some datasets and some benchmarks specifically for French institutions, and they have an easy dataset and a hard dataset, so that’s something that we’re going to be looking at.

Amanda French

Thank you so much for your time, Justin.

Adam Buttrick

I am probably the most excited about this interview, of all the readers!

Justin Barrett

Thank you very much.

About ROR and registry curation

The Research Organization Registry (ROR) is a community-led registry of open, sustainable, usable, and unique identifiers for every research organization in the world.

ROR is a cross-organization collaborative led by California Digital Library, Crossref, and DataCite. Each organization provides input on decisions and overall strategy that support the growth and sustainability of ROR by engaging in outreach to funders, potential adopters, and other stakeholders in their respective networks.

The ROR registry includes IDs and metadata for more than 105,000 research organizations and counting. ROR receives regular input from its global community of users and stakeholders about records that should be added to or updated in the registry. ROR also seeks and incorporates input from the broader community about policies and processes for maintaining registry data. We review and process this feedback through a centralized process and according to metadata policies established and maintained by ROR’s curation advisory board and metadata curation lead. All requests are publicly visible in an open Github repository so that the process is as transparent as possible. New registry releases are published on a rolling basis, approximately once a month.

Since officially launching its new curation model in early 2022, ROR’s volume of curation activities has been increasing steadily. In order to keep up with this growth, we are looking to scale resourcing on the ROR team accordingly, beginning with hiring short-term contract support to help maintain the pipeline of requests. Contract support will better equip ROR to be as responsive as possible to requests, to work with community members interested in submitting unique types of requests such as bulk lists of organizations, and to pursue special strategic projects such as the reconciliation of Crossref’s Open Funder Registry with ROR. This short-term support will also allow us to assess bigger-picture and longer-term needs with regard to curation staffing.

The project description and application instructions follow below.

Curation Contractor: General responsibilities

• Triage new curation requests using Github-based and command-line tools
• Evaluate accuracy and validate requests by researching information in ROR, on organizational websites, and other sources such as Google Scholar
• Review bulk requests as assigned by the metadata curation lead
• Provide recommendations to curation lead and curation advisory board about how requests should be processed
• Provide input to curation lead and curation advisory board to help improve the curation process and registry data quality
• Provide support to curation lead and curation advisory board if troubleshooting is needed

Contract deliverables

Curation backlog management and processing

Carry out Github- and command-line-based workflow to triage new curation requests and keep backlog small. Evaluate metadata accuracy and complete requests by researching information in ROR, on organizational websites, and other sources such as Google Scholar. Provide recommendations to curation lead and curation advisory board about how requests should be processed

Project support

Review bulk requests as identified and assigned by the metadata curation lead. Provide support to curation lead and curation advisory board if troubleshooting is needed

Data analysis/metadata Q&A

Identify opportunities to improve ROR metadata quality and address metadata quality issues at scale.

Skills and qualifications

• Familiarity with Github issues and projects
• Familiarity with command line operations
• Ability to use basic functions of the ROR API (we can provide training)
• Ability to perform intelligent searches on the ROR website (multiple queries etc) and related authority sources (e.g. Wikidata, ISNI, ORCID)
• Excel/spreadsheet data analysis
• Attention to detail
• Professional proficiency in English and at least one other language
• Ability to search and analyze information about research institutions and scholarly publications
• Knowledge of research/publishing landscape and persistent identifiers for scholarly communications desirable but not required

Position details

This is a part-time contracted position (up to 20 hours a week) for approximately three months. This position will be in contract with Crossref but will be accountable to the three organizations that operate ROR (Crossref, California Digital Library, and DataCite).

All work will be performed remotely. The contractor will have no set work hours.

To apply

To apply, send a resume, statement of interest, and rate schedule to jobs@ror.org. Proposals will be reviewed until the position is filled. Review of proposals will begin September 15.

Equal opportunity statement

ROR, and the collaborating organizations behind it, California Digital Library, Crossref, and DataCite, are committed to non-discrimination and making opportunities available to anyone without regard to race, color, religion, sex, pregnancy or a condition related to pregnancy, sexual orientation, gender identity or expression, national origin, ancestry, age, physical or mental disability, genetic information, veteran status, uniform service member status, or any other protected class under applicable law.

Open Funder Registry to transition to the Research Organization Registry (ROR)

Today, we are announcing a long-term plan to deprecate the Open Funder Registry. For some time, we have understood that there is significant overlap between the Funder Registry and the Research Organization Registry (ROR), and funders and publishers have been asking us whether they should use Funder IDs or ROR IDs to identify funders. It has therefore become clear that merging the two registries will make workflows more efficient and less confusing for all concerned. Crossref and ROR are therefore working together to ensure that Crossref members and funders can use ROR to simplify persistent identifier integrations, to register better metadata, and to help connect research outputs to research funders.

Just yesterday, we published a summary of a recent workshop between funders and publishers on funding metadata workflows that we convened with the Dutch Research Council (NWO) and Sesame Open Science. As the report notes, “open funding metadata is arguably the next big thing” [in Open Science]. That being the case, we think this is the ideal time to strengthen our support of open funding metadata by beginning this transition to ROR.

Comparing the features of ROR and the Funder Registry

Let’s look at some of the major similarities and differences between the two registries, including their history, features, scope, and usage, since there are important nuances and distinctions that are helpful to understand.

Overview

History

The Open Funder Registry was launched as FundRef over a decade ago to enable the community to cite research financing and assert it within the scholarly record, acknowledging the organizations granting their support. Elsevier generously donated the seed data for the Funder Registry and has managed its curation for the last ten years, while we have maintained the technical operations and promoted community adoption of the Funder Registry.

The Research Organization Registry (ROR) was introduced in 2019 by the California Digital Library, DataCite, and Crossref to enable the community to cite contributor affiliations and assert them within the scholarly record, acknowledging the organizations that housed or performed the research. Digital Science generously donated the seed data for the Research Organization Registry from its Global Research Identifier Database (GRID) initiative, and Crossref, DataCite, and the California Digital Library have contributed labor and resources to turn ROR into a mature, independent, freely available offering.

Scope

One key difference between the registries is that ROR has always included funding organizations, and ROR records have always included mappings to Funder IDs where available, while the reverse is not true: the Funder Registry includes only funding organizations, not other kinds of organizations, and Funder Registry records do not currently include mappings to ROR IDs or other identifiers. It therefore makes sense to expand ROR’s initial contributor affiliation use case to include the function of identifying research financing.

Usage

More Crossref members use Funder IDs than use ROR IDs, to be sure. You can see from the table above that the number of Crossref members using Funder IDs in Crossref records is higher by almost a factor of 10 than the number of Crossref members using ROR IDs in Crossref records. But note too that the current rate of adoption is far higher for ROR than it is for the Funder Registry. Since January of 2022, we’ve seen a gratifying number of publishers and service providers beginning to use ROR identifiers for contributor affiliations in Crossref. In the last year, the number of Crossref members depositing ROR IDs has increased by 356%, while the number depositing Funder IDs has increased only by 12%. As evidenced by its ballooning API traffic, too, with more than 20 million requests last month,³ ROR is clearly being used by many scholarly research systems for many purposes. The more systems that use an identifier, the more valuable that identifier becomes as a vehicle for exchanging information.

Even though ROR’s primary use case has been to identify contributor affiliations, ROR is in fact already being used by funders. Nineteen funding organizations are depositing ROR IDs in their grant records with Crossref to denote principal investigator affiliations,⁴ and, following a meeting of the Crossref Funder Advisory Group last month, all eighty funder members are primed to start using ROR IDs to identify themselves in grant records. DataCite has allowed ROR IDs as a funding identifier since 2019⁵, and while there are currently over 877,000 DataCite records that use Funder IDs to identify funders,⁶ there are also over 161,000 DataCite records that use ROR IDs to identify funders.⁷

Tools and services

Both the Funder Registry and ROR offer open data and open source code, but we think that ROR’s suite of free and open source utilities (some of which were developed by Crossref staff) gives it a competitive advantage. We know that publishers and their service providers have ongoing challenges in collecting and matching funding information from authors and in validating Funder IDs. With ROR’s extensive toolkit, publishers and their technology providers who adopt ROR will be in a much better position to improve the accuracy of funding acknowledgements in metadata, which can in turn enable the development of reliable analytics, tools, and services for funders, regulators, research facilities, and the public.

Crossref has built tools based on OpenRefine for both the Funder Registry and ROR: the Open Funder Registry Reconciliation Service and the ROR Reconciler are both useful ways to clean messy data. ROR, however, also offers a much-used API endpoint that helps match organization names to ROR IDs, and several third parties have also developed and shared open source matching tools and services for ROR. Crossref and ROR are also collaborating on new strategies for affiliation matching that will be able to match funding references.

Community engagement models

The Funder Registry has been curated for over a decade through time and expertise generously donated by Elsevier. ROR offers more transparency and community involvement; it is openly governed by Crossref, DataCite, and the California Digital Library and is advised by a global network of community stakeholders through its Steering Group and Community Advisory Group. ROR is openly curated and is aided by a global Curation Advisory Board of volunteers.

Summary

For all of the above reasons, then, we believe that in the long term ROR will serve the community better as an identifier for funders. In a future post, we’ll do an even deeper dive into comparing the Funder Registry and ROR, comparing the metadata and data in each registry and giving statistics on funder assertions in our metadata.

What will this mean for you?

The many organizations whose tools, services, and workflows have been architected to use Funder Registry IDs will find this transition a challenge, and we don’t want to make light of that issue. Over the last ten years, we have encouraged the community to adopt Funder IDs, and the community has demonstrably recognized the benefits of doing so. Publishers have put a great deal of time, thought, and effort into collecting funder data and including it in Crossref metadata, and they have built internal reports and workflows around the Funder Registry. Both Crossref and ROR are committed to making the transition from the Funder Registry to the Research Organization Registry as simple as possible for those who have adopted the Funder Registry.

If you are not already using the Funder Registry and are planning to begin standardizing funding data, we recommend that you use ROR to identify funders. If you are currently using the Funder Registry in your systems and workflows, don’t worry! In the short term, and even in the medium term, Funder IDs aren’t going away. Eventually, however, the Funder Registry will cease to be updated, so any new funders will only be registrable in Crossref metadata with ROR IDs. Legacy Funder IDs and their mapping to ROR IDs will be maintained, so if Crossref members submit a legacy Funder ID, it will get mapped to a ROR ID automatically. Note, too, that Crossref is committed to maintaining the current funder API endpoints until ROR IDs become the predominant identifier for newly registered content.

In short, if you are already using Funder IDs, you can and should continue to do so. However, we do recommend that you begin looking at what it will take to integrate ROR into your systems and workflows for identifying funders. Think of it as warming up before a workout: it’s time to start swinging your arms and stretching your hamstrings.

We face challenges in this transition, too. Of these, we think the largest will be (1) completing the reconciliation work involved in mapping Funder IDs to ROR IDs, and (2) overhauling Crossref’s schemas, APIs, and deposit tools to support ROR IDs in all the ways we currently support Funder IDs. We’ll discuss both of these challenges in future blog posts, but it’s worth saying that any challenges pale in comparison to the benefit of enabling the whole community to use a single open identifier in multiple places in the scholarly record.

Tell us what you need!

We want to hear from you. You can use our Community Forum talk to us about the Crossref Funder Registry, and you can join the ROR Slack to talk to the ROR team and community. You can also contact Crossref via our request form or email ROR at info@ror.org, and you can attend online Crossref events and ROR events to get updates from us and ask us your questions.

One of the major messages we’re already hearing from funders and publishers is expressed in yesterday’s post on open funding metadata: “While many concluded that there was still a long way to go to solve the many technical challenges related to funding metadata, attendees were unanimous on its importance.” We look forward to beginning this important work together.

Amanda French

Hello, and thank you for participating in this case study interview. Can you tell us your name, title and organization?

Chris Hartgerink

Hi, everyone, I’m Chris Hartgerink from Liberate Science. I’m the CEO and founder there, and before I was a practicing researcher in meta-research.

Amanda French

And tell us about Liberate Science. When was it founded? What does it do? What makes it special?

Chris Hartgerink

Well, Liberate Science was founded back in 2019 when I left academia. I wanted to keep working on improving research, but not within the academic institution, because I tried that, and even though it might work for some people, it definitely didn’t work for me. I always felt like I was working towards not having a loss instead of towards wins. What’s different about Liberate Science really is that it tries to stay as independent as possible so that we can grow serious alternatives to all aspects of doing research. Right now our focus is on a serious alternative to the big publishers and an alternative to the article format. And so we built ResearchEquals, which was the first modular publishing platform. If you like that, it’s quite special.

Amanda French

Wonderful. What are these alternative formats in scholarly publishing?

Chris Hartgerink

Alternative formats are really nice, because they vary in both size and content. Of course, the book is the largest entity of publishing, and the article is a smaller one. One alternative format is the micropublication, which is a small article, but it is still an article. Then there’s modular publishing, which can be as big as a preprint, but it can be as small as a conference abstract. And then there’s also nanopublications, which are the smallest entity I know, where it’s just the statement that a certain variable positively predicts another variable, and that is then one publication.

Alternative formats are not necessarily paper-based. The nice thing about them is that they’re creating diversity in the publishing space. We’re used to reading text, but why not also have podcasts be part of the scholarly record? And videos, and study materials, and blogs? We’re really seeing much more diversity in the outputs and that increased diversity in what gets published will be good for the assessment of researchers as well.

Amanda French

Fascinating. So when and where did you personally first hear about ROR?

Chris Hartgerink

I heard about ROR for the first time when it was still in development. In following along over time, it just made sense to me: ROR is to organizations what ORCID is to individual researchers. Of course we want to know where people come from, what their affiliation is, and have that be entirely unambiguous. And so I followed along on GitHub a bit and saw some of the discussion happening there. I really like this community-based curation model. As with any open source work, it always takes a bit of time. But I would say I’ve known about ROR since 2019.

Amanda French

We calculate ROR’s official launch date as January 2019, which means that ROR’s fifth anniversary is coming up in January 2024.

Chris Hartgerink

Anything planned?

Amanda French

Yes, there will be events! We’re beginning to plan. Any suggestions? What would you like to see happen at ROR’s fifth anniversary?

Chris Hartgerink

I would be very interested in some creative celebrations, something that really marks the moment, because I think you and the team have achieved a lot in those five years. And I think that’s worth literally celebrating. So maybe a virtual party?

Amanda French

I love it. Sounds amazing.

Chris Hartgerink

There are all these artists and drag shows and performances on Zoom. I don’t know whether they still do it.

Amanda French

At Crossref, we once had an origami lesson on Zoom at an all-staff meeting, which was wonderful. Although that was maybe more meditative than celebratory. And speaking of creativity, I so admired your live streaming of your integration of ROR into ResearchEquals! I thought that was something that more people should emulate. And it was so useful for us to see. Can you tell us about that, and about why you decided to integrate ROR into ResearchEquals?

Chris Hartgerink

ResearchEquals is a modular publishing platform, so we have a bit of a different situation from regular manuscript processing. With us, every researcher or author can make their own profile. And it’s not that they create an account and then go through a whole submission process and only then somebody else says, “Okay, now it gets published.” On ResearchEquals, you as the author or authors get to say, “Okay, now this becomes public.” It’s a bit more like a repository in that sense.

One of the things that came up was “What metadata do people want to add to their profile?” We started out with things like author names, their pronouns, their website, and a bio, but one of the things that was coming up was “How can we discover other people?” Authors also started saying, “We have a certain affiliation, and we want to be able to add that.” So then I started looking into that, and I remembered ROR! I was already inclined to integrate ROR anyway.

This way, we have the option in the future to also say, “Well, I’m not interested in specific authors, but I’m interested in specific organizations.” Integrating ROR just makes it very easy to match all these authors to the same institution. And it also lets you get all the metadata related to an institution so that you can display institutions visually and add more information, like when the institution was founded and where it’s located, all this information that you wouldn’t get from just a raw string.

Plus, there’s the risk, as you also know, that I might fill out my affiliation ever so slightly differently than someone else would. ROR was also a cost-effective way to implement affiliations. So all in all, there were many pluses in the pro column for ROR.

Amanda French

And why did you decide to live stream it?

Chris Hartgerink

The live streaming was actually very fun. I was thinking about integrating ROR, and then I thought, “Okay, but I have so many other things on my plate. How do I actually plan this properly?” It was a challenge to myself. It was four weeks, four two-hour slots, where I said, “Well, I’m just going to plan those two-hour slots for every week, and then I’m going to get it as far as I can.” Even if nobody watched, the live streaming would help me to plan and to have this experience of rubber duck programming, where you try to explain to your rubber ducky what you’re actually doing. It ended up being incredibly helpful. In the live stream itself I got some feedback that helped me along the way and made me look at things differently.

And so yeah, the idea was for self planning, but also to take the open source aspects of ResearchEquals to the next level. Or maybe not the next level, because other people have done this, but I thought this would be a fun exercise to try. And I was surprised how many people actually ended up watching the stream. I mean, it wasn’t thousands, of course, but I think one of the videos actually got around a hundred views over time. Not very long views, but still.

Amanda French

Right! I thought they were fascinating, of course. I learned a lot from them. And there were moments during the live streams where I kept wanting to help you – and then not doing it, to be honest! Because I thought, “No, I want to see if our documentation is good enough so that Chris can figure it out, and if they can’t, then I’m going to go and change it.” So I did that several times, actually, after watching you struggle a little. I apologize for not helping you when I could have, just because it was so useful to see how our documentation could be improved.

Chris Hartgerink

Well, you’re off the hook.

Amanda French

Thank you. And how has that integration been? Do you find organizations that are not in ROR that people are entering?

Chris Hartgerink

We ended up implementing it in such a way that people can only enter organizations from ROR. We include the ROR ID in our database and not the raw strings, because otherwise we’d have this issue still, which is the exact one we’re trying to solve, of needing to disambiguate the organizations. So when people add an affiliation, they all use ROR. I have not heard any feedback that people can’t find their institution. Sometimes it might be under a slightly different name than they expect, especially if there’s a bilingual name or if it’s known under different names.

One of the things we have heard is a limitation, because we only had eight hours to implement, is that at this point we only allow one affiliation at any given time. People have said, you know, “I have multiple affiliations, I would like to add them,” so we’ll be adding that at some point. It’s always a matter of resources. The other thing that we’re going to be adding are dates for each affiliation so people can also keep track of their past affiliations in their profile. That way it’s easy to see how many people were at these organizations over time, so if you’re not interested in the person per se, you can track how many people are using ResearchEquals at the organizational level. Of course, that doesn’t go into metadata, but that’s something we want to offer to institutions.

Amanda French

Great. And are you using the ROR API or the ROR data dump?

Chris Hartgerink

For ingestion, we use the ROR API, simply to make it as up to date as possible. We also store organizational information whenever people add it in, so it’s more efficient. I think that’s been very helpful. Of course we try to keep the queries to a minimum, so it’s only when people are searching that ROR gets asked “Hey, does this exist?” And then we store that information in our own database, and we may periodically double-check whether everything is still up to date.

Amanda French

And do you send those ROR IDs out to any other systems, like DOI registration agencies such as Crossref? Or do they just stay in your system?

Chris Hartgerink

Whenever I have an affiliation in my account, if I publish a module on ResearchEquals, it immediately gets added to my DOI metadata at that time. So if I switched my affiliation, job-wise, and I switched it in ResearchEquals, then at that point all the new DOIs would also have the new affiliation in there. All the old ones, of course, would have the old one.

What we don’t yet do is do the backlog. If you’ve authored with ResearchEquals before, pre-ROR integration, you don’t have this info yet. If somebody is reading this and they want that updated, then we can always regenerate the metadata for the DOI.

Amanda French

Gotcha. What challenges did you run into as you were integrating ROR into your system in four two-hour blocks?

Chris Hartgerink

I think the main thing was simply learning some of the quirks around how to interface with ROR directly. I think that’s true for any API integration. I write my documentation differently from the way somebody else does, so you have to sort of learn the language a bit to know where to find things. The main thing that was confusing or was a difficulty was this idea that the URL is the ID. I thought it was going to be just the unique string part.

Amanda French

Right.

Chris Hartgerink

That was a bit of a mindset shift. But beyond that, I think it was fairly straightforward. And most of the time, I was actually busy trying to fit all the design elements in the right place and make sure that when people click here and there that it works.

Amanda French

The interface design is a big issue, because there’s lots of different ways to implement ROR in terms of UI.

Chris Hartgerink

Yeah. And actually I think after the first two sessions, so after four hours, the integration was pretty much complete. And the remaining four hours were spent simply, you know, moving stuff one pixel to the right to make sure that it displays properly.

Amanda French

I had actually forgotten this until you mentioned it, but one of the things that I changed after watching your live stream was the part of our documentation about the preferred form of the ROR ID. There’s actually a lot of discussion on the ORCID GitHub about why our preferred form is the entire URL, and one of the changes I made was to add that reasoning into the documentation. Essentially, our sense is that when the URL is used as the ID in metadata, it’s more likely to continue to resolve. And that’s the key thing that you want from a persistent identifier.

I also put in the documentation that while we think the entire URL is the best form for the ROR identifier, the ROR API does also recognize just the unique string. A ROR ID is https://ror.org plus nine characters, and if you just use that nine-character string, that will work in the API.

Chris Hartgerink

I mean, I think it makes sense, because then ultimately, in the code, that means whenever we want to link out, we don’t need to add all of that. Yeah, I think in that sense, it’s very worthwhile.

In terms of difficulties, I think for me, it’s also always that there’s just so many systems to integrate with, and they all have their own quirks. If you have a big team, you could just say “You’re the Integration Specialist for this, and you for this.” But we’re a very small team, and actually I’m the one responsible for these integrations, so I need to remember the quirks of ROR, remember the quirks of ORCID, remember the quirks of Crossref and whatever other integrations we have. That’s always a challenge to keep space for all that in my head.

But, you know, if the documentation is clear, that improvement is also going to be helpful to me in the future, because I Google everything. Somebody once told me that the documentation is a reflection of the state of the community. Because if feedback doesn’t get taken up into the documentation, then that’s an indication that they’re not really listening.

Amanda French

Oh, you’re so right. I think I got up at 5am or something to watch your livestream, and the minute it ended, I thought, “Okay, this documentation is changing.” I absolutely believe that, that there really needs to be a virtuous circle of getting feedback from community and immediately putting that into the documentation. Whenever we get a support request, I think, “If one person has this question, and they’ve overcome all the barriers to finding out how to ask it, has gone to the trouble of starting an email or posting in the Slack or whatever, then clearly other people have that question, too.” Any time one person has an issue, it’s almost certainly the case that there are fifteen other people who have that issue who just left the page and never communicated with you about it. So yes, I completely agree about that. And again, thank you so much for all of the free user testing you gave us.

Chris Hartgerink

My pleasure. I was doing it anyway. So yes, that’s good. Another benefit of live streaming.

Amanda French

Yep. So what do you hope ROR does in the future? Do you have any feature requests or suggestions for future direction?

Chris Hartgerink

Well, I think for me, one of the things that I find very undervalued and that I’ve also come to enjoy a lot is maintenance. So in that sense, I don’t have any shiny new feature requests except for stability and reliability. I think that very often when something is maintained, a lot of people feel like it stagnates if there is not anything new. But I very much like the idea of ROR being a very reliable, stable service where I don’t have to worry about the integration breaking anytime soon, or if it were to break, that there’s sufficient time to handle it. So in that way, my biggest feature request is stability.

A nice thing about ROR is that it is living, breathing – with the continual releases, there’s new organizations being added, making it more worthwhile without us doing anything. As the data get richer and the usage gets richer, I think that’s really where the added value is. I’ve seen other infrastructure providers try to create too many projects, which sometimes is a disservice to the core infrastructure. So I think that would be the primary thing. If anything, I guess the ease with which people can submit for community curation or participate in assessing requests is something to continuously evaluate.

Amanda French

Yep. Good feedback. I think one of the issues is that we’re specifically looking for people from Africa and Asia at the moment. I think if we recruited only from the US and Europe, we would get lots and lots of curators, but so much of what we need from our community curators right now is regional expertise. We need to recruit from Africa and Asia so that we can understand the organizations in that area. But yes, I agree, maybe we should work on clearer guidelines about that.

Chris Hartgerink

And if there’s anything we can do, if we can add a link out to say, “Hey, are you missing an institution? Go here to add it.” Or, you know, if people are interested to curate, that people can see that, always happy to discuss that. But I can imagine that having only European and North American universities, departments, etcetera, included is going to be really what’s going to be setting ROR apart.

Amanda French

What else would you like to say about ROR? Or about ResearchEquals? Or about Liberate Science?

Chris Hartgerink

I think my main thing is I want to have a big festival when you turn five, because it’s a momentous time. You turn five first, and then Liberate Science turns five later next year. So that’s gonna be fun. And to whomever is reading this, if you’re interested in modular publishing, check out ResearchEquals. If it’s too intimidating, we also do cohort trainings where we get a group of people together to just learn over several weeks, in predetermined hour-long slots, where people can also then you know, take the space and just say, okay, that’s when I’m going to learn this. And never outside of it.

Amanda French

“Time-boxing” is the phrase that I’ve heard for that. But for me, time-boxing only works when there’s an external constraint.

Chris Hartgerink

That’s the nice thing with live streaming.

Amanda French

Exactly! But that also takes a lot of courage. So maybe some time I will emulate your courage in doing that. Anyway, thank you so much for speaking with us!

Chris Hartgerink

Enjoy the rest of your day, and then have a nice rest of your week.

• ROR usage continues to skyrocket, with the ROR API seeing more than 23 million requests every month.

• ROR integrations are continuing to grow, and we’re showcasing specific implementations in a new case study series on the ROR blog.

• The number of ROR IDs available in Crossref and DataCite is steadily increasing, which also feeds downstream sources like OpenAlex and DataCite Commons.

• ROR IDs are being recommended in a growing set of national PID strategies and in ongoing discussions and recommendations related to the August 2022 OSTP memo, including in feedback submitted in response to United States federal agencies’ public access policies (see, for example, the Association for Research Libraries’ response to the NIH draft policy).

In addition, we’ve published 10 new data releases since the beginning of the year in response to community feedback about registry additions and updates, and as part of ongoing metadata quality checks on existing registry records. ROR now includes more than 105,000 IDs and associated metadata records, and new registry updates are available on a rolling basis, approximately 1-2 times per month.

All of this progress is happening - and will continue to happen - because organizations are supporting ROR’s growth, guiding its development, and investing in its long-term sustainability.

Supporting ROR, as a community

As we’ve described previously, ROR’s sustainability model is based on a shared resourcing commitment by ROR’s three operating organizations - California Digital Library, Crossref, and DataCite - to fund ROR’s core operating expenses over the long term. This commitment ensures a baseline level of funding for ROR, enables us to make the service freely available for everyone, and minimizes dependence on unpredictable or time-limited sources of funds.

To supplement this baseline resourcing, ROR continues to receive outside investment by stakeholder organizations. These additional funds allow ROR to accelerate and scale our work and activities, and to invest in strategic time-limited projects. Community funding also showcases the importance of investing in core open infrastructure that all can benefit from regardless of their ability to pay for it.

Core infrastructure for other core infrastructure 

The latest example of community investment in ROR comes from ORCID. Last week, ORCID announced that it would be increasing its financial contribution to ROR to support ROR’s continued growth. ORCID is a long-time supporter of ROR, having been involved in the early planning stages that defined the initial vision for ROR, and active in community activities ever since then through our community advisory group and steering group. ROR is now the core organization identifier supported in ORCID and makes it possible for ORCID to normalize affiliation information against an open and interoperable identifier without being reliant on proprietary data. As ORCID’s Executive Director Chris Shillum explains, ROR plays a key role in the organization’s overall strategy: “Having a reliable way to identify organizations helps ORCID users and members in multiple ways, as a growing number of use cases clearly demonstrate. ORCID commends the commitment made by CDL, Crossref and DataCite in 2022 to jointly fund ROR’s core operating costs. As a fellow scholarly infrastructure organization, we at ORCID feel it is also our responsibility to contribute to ROR’s sustainability on behalf of our community.”

Supporting ROR through global coalitions

ROR has also been receiving investments from organizations, library consortia, and national research offices contributing to sustainable open science infrastructure since being selected by SCOSS in November 2022. This selection identifies ROR as essential open infrastructure, and as part of SCOSS’s fundraising campaign, organizations are encouraged to invest in noncommercial scholarly infrastructure initiatives to advance open knowledge. Nine months into the SCOSS campaign, we wanted to acknowledge the organizations that have stepped up to support ROR and highlight some of their reasons for investing in the open infrastructure that ROR provides and enables.

• Dutch Research Council (NWO) (Netherlands)

• Consortium of Swiss Academic Libraries (CSAL) (Switzerland)

• Canadian Research Knowledge Network (CRKN) (Canada)

• Council of Australian University Librarians (CAUL) (Australia)

• Royal Danish Library (Denmark)

• SLUB Dresden (Germany)

• French National Funds for Open Science (FNSO) (France)

For these organizations, ROR represents an opportunity to advance strategic goals around identification and tracking of research. “Persistent identifiers are key to a robust scholarly infrastructure that makes the research process more efficient, clear, and meaningful,” said Clare Appavoo, CRKN Executive Director. “CRKN is a proud supporter of persistent identifiers through the ORCID Canada and DataCite Canada consortia, and we’re pleased that ROR is a recipient of our member contributions to SCOSS in this funding round.”

Open identifiers are especially important in the changing landscape of research dissemination. Angus Cook from the Council of Australian University Librarians described the importance of supporting infrastructure that supports open access:

“As consortiums and libraries expand services through publishing activities, in addition to providing access to content, supporting infrastructure that supports open access becomes increasingly important. One aspect of agreement management that becomes more critical as consortiums transition their agreements to open access, is the correct identification of participating institutions. Correct identification of institutions is a vital part of the workflow in ensuring authors are affiliated with the correct entity. As part of CAUL’s support for OA agreements and supporting infrastructure, we are keen to see open and accessible identifiers, such as ROR, being adequately resourced. Identifiers, like many standards, should be openly accessible and freely available.”

Why CDL, Crossref, and DataCite are supporting ROR

ROR is also a key component of each of its operating organization’s strategic objectives. ROR was identified by Crossref members as a top priority back in 2019, and ROR IDs play an important role in Crossref’s vision of the research nexus and its aims to make scholarly information easier to find, assess, and connect. For DataCite’s strategic focus on identifying and connecting knowledge, ROR IDs are key to establishing connections in DOI metadata to enable meaningful insights about how research is conducted and used. For CDL, investing in open metadata for affiliations aligns with a multi-faceted approach to make UC research - including the underlying metadata - more openly available. For example, identifying UC research to be made openly available requires accurate identification of UC-affiliated authors.

A commitment to open

As an early and continued signatory to the Principles of Open Scholarly Infrastructure (POSI), ROR is committed to following best practices to ensure its long-term openness, availability, and sustainability. ROR’s operations are specifically structured to maximize the openness and availability of its data and protect against risks, vulnerabilities, and unpredictable circumstances that can befell other initiatives that don’t have similar safeguards in place. This is an especially crucial time to understand the importance of what it means to be open, and we are proud to be in the company of other POSI-aligned infrastructures committed to following similar practices.

We are grateful to all of the organizations that have been supporting ROR in one way or another over the years, and in the years to come.

Amanda French

Hello! If you could, please start by telling us your name, title, and organization.

Michael Parkin

My name is Michael Parkin. I’m a data scientist at EMBL-EBI, the European Molecular Biology Laboratory’s European Bioinformatics Institute. I work on the Europe PMC team.

Amanda French

Can you tell us about the relationship between EMBL-EBI and Europe PMC?

Michael Parkin

There’s basically three levels. The parent organization is EMBL, the European Molecular Biology Laboratory. It’s headquartered in Heidelberg, Germany, and consists of six sites around Europe. There’s a big research center in Heidelberg, a synchrotron facility in Hamburg, a research institute outside of Rome, another one in Grenoble, and one in Barcelona. The sixth site is EMBL’s European Bioinformatics Institute (EMBL-EBI), based in the UK. All of these sites combine to make EMBL, which is a large, intergovernmental, not-for-profit organization, and Europe’s life sciences laboratory.

A lot of the EMBL organization is doing primary research and wet-lab work. Here in the UK, we are mostly on the data side of things. EMBL-EBI is split into two parts. Half of the organization runs data services and the other half is doing research, so some of my colleagues are bioinformaticians. I work in the services side, where we have many teams that look after life science databases, which are freely-accessible to anyone with an internet connection. For example, you run a research study that generates a lot of genomic data, and you want to put that data somewhere. EMBL-EBI will have a resource for that and an associated team that looks after it. Most of the teams manage these big biodata type resources.

My team is a little bit unusual because we are the literature team; literature is our data point. Our team maintains Europe PMC, which is a life science literature database. So there are a lot of similarities to the other kinds of big data that are stored at EMBL-EBI, but we’re a bit different because we’re focusing on the literature outputs rather than the data outputs that are generated by research. We are funded by EMBL itself, as well as 37 other funding organizations, mostly in the UK, but also across Europe and more globally.

Amanda French

The volume of scholarly literature today makes it essentially big data.

Michael Parkin

Yeah, it’s gotten increasingly bigger over time. The biggest data source at Europe PMC is PubMed. We have a faithful copy of everything that’s in PubMed. That’s approaching 40 million abstracts, in the mid-30s at the moment. And really the genesis of Europe PMC was around collaboration and data sharing with the NCBI, the National Center for Biotechnology Information, which hosts PubMed and PubMed Central, a database for full-text copies of journal articles. I think there’s over 7 million of those. So EMBL-EBI is really well positioned. We have a lot of computer power and data storage, either on site here outside of Cambridge or within the UK. And the volume is significant. So it does fit well within the scope of these databases, and one of the nice things, of course, is they’re all public access. It’s just kind of really a nice thing. The idea is that it’s for the research community. And so at the core, this is a public good.

Amanda French

Wonderful. When and where did you personally first hear about ROR?

Michael Parkin

I started at EMBL-EBI in 2016, and a couple of years into my time here I was getting interested in PIDs, and there was the opportunity to go to the PIDapalooza Conference. I went to the one in Girona in 2018, and I remember there was an open session, and I think it was someone from Crossref who gave a talk to say “watch this space” in terms of organizational identifiers. I don’t think the name “ROR” had been chosen at that point. There were actually many presentations at PIDapalooza 2018 about organization identifiers, whether from a librarian’s perspective or publisher’s. But I think there was just a few minutes’ long talk to say that Crossref and ORCID were working on organization identifiers.

Amanda French

You are in fact correct! I was not there, but having studied the history of ROR, I know a great deal of the technical work that did become ROR shortly thereafter was going on then. ROR’s official launch date is January 1st, 2019, and it was the Crossref technical team that really did a lot of that work to stand that up. So I’m perfectly certain that in 2018 they were working on it, and I do think the ROR name was chosen in late 2018 or something like that.

Michael Parkin

I feel like I would have remembered some lions!

Amanda French

You’re talking about the pre-lion period!

Michael Parkin

There were a lot of talks at the conference about the use of organizational IDs, and the idea of something coming from organizations like Crossref and ORCID, who we know a lot about and often collaborate with, was a really cool thing. I was kind of interested to see where it went, so I guess that that’s where I first heard about what became ROR. Perhaps that was the first announcement to the public.

Amanda French

Well, there were working groups from 2016 to 2018, really about three years of discussions and working groups and plans to try to stand it up, and then the official launch in 2019. And now here we are! Who were the primary advocates of implementing ROR at Europe PMC?

Michael Parkin

There are two main angles to this. As I mentioned, Europe PMC is funded by funders of life science research, and funders typically have a strong desire to track the outputs of their funding. They want to see the impact their funding has. And one aspect of that is to look at journal publications. That’s often the way scientists communicate their results, and those results are an outcome of the funding that they were awarded by a particular grant. Part of the puzzle of that is, “Well, which institutions does this funding go to?” That’s a key thing to keep track of, and it gets very, very difficult to track when affiliations in publications are captured just as strings of text; it’s very messy. So there’s a desire from our funders that Europe PMC has some way of disambiguating institution names as a piece of the impact assessment puzzle. That’s one of the reasons we want to be able to do closer inspection of institutions via IDs.

The other side is that EMBL, the organization that we’re a part of, produces a lot of its own research, and EMBL wants to know where EMBL-associated researchers are publishing, and especially, who are they collaborating with? That’s a really key thing for us. It’s, “Where are the networks between EMBL researchers and other institutions around the world?” And again, one way that gets measured is by looking at journal articles, finding which ones have been published by groups in our organization and the other institutions they worked with. Again, really hard to do when these affiliations are just strings of text. So there’s also a desire from our organizational side, to be able to have better information about which institutions are which, and making networks and cross-connections between the two. We do a lot of that with ORCID, and we want to do the same with ROR and institutions. Those are the two key drivers, one side for the funders for research impact assessment, and one from our own organization, knowing who’s doing the research and where the collaborations are. Those have been the main use cases for us.

Amanda French

Wonderful. I’m going to ask a follow-up question, because I have heard from others that they are particularly interested in this collaboration question, finding out who their researchers are collaborating with. Why is that an important thing that people want to know? What are you going to do with that information once you can find it out?

Michael Parkin

I don’t know too much about the details, because our team generates the data for others at EMBL to digest, and EMBL uses this information in a number of ways. But essentially, it showcases the collaborative nature of our work to our funders. EMBL helps to create links and connections between different research areas in the life sciences, so being able to show the number, scale and scope of our collaborations is one way of demonstrating our impact. I think that’s where EMBL sees this collaboration information as being valuable.

Amanda French

That’s helpful, actually. It’s something that I’ve heard some people who are implementing ROR say, that that’s part of why they’ve implemented ROR, is to track research collaborations. Universities want to track their own institution’s output, which is surprisingly difficult, but they also want to know who their researchers are collaborating with. So I think it’s similar to what you’re talking about: Are our researchers in this American institution collaborating only with European researchers? Or are they, you know, really taking a global view and collaborating as much as they could be or should be with researchers in Asia or researchers in South America. My sense is that that’s why they want to know that.

Michael Parkin

I expect these collaborations are usually not something individual researchers worry about too much. But at the higher strategic level this becomes an important question to address. Maybe eventually it does feed down to the researchers, and it will steer the collaborations: “Well, you know, we’re not really doing a lot of work with, say, South America, maybe some project around there or some collaborative network would be good.” Publications being just one thing – it helps you see whether that collaboration is fruitful or not.

Amanda French

Describe for us what you’ve done with ROR at Europe PMC. I think you’ve done several things, actually.

Michael Parkin

I’ll talk more about the more recent one. But the initial work we did, which is already listed on the ROR website, was part of a European Commission-funded project called FREYA, which was a successor to the THOR project. They’re committed to Norse mythology! The FREYA project was all about PIDs and PID interconnectivity, and as part of that we were looking at publications. It’s important to mention that in Europe PMC we have our publications world, which is most of what we do, and we also have the grants world, which is exclusively for the Europe PMC funders. I’ll just make that distinction now, because I’ll talk more about the grants side of things.

But the initial work we did with ROR was on the publication side of things, and it was exactly this idea that our organization, EMBL, wanted to have publication data about EMBL sites, how many papers they were publishing and which organizations they were collaborating with. We had a fantastic intern on the team at the time who built a machine learning model that would look at the affiliations in publications and pick out ones that had an EMBL site affiliation on them. As I mentioned, there’s a few of them around Europe, the sites outside of Rome, in Grenoble, in Heidelberg, in Hamburg, and luckily there’s a ROR ID for each of those, which is fortunate, because I can assign papers to individual institution locations.

Amanda French

And if there weren’t a ROR ID, you could request one.

Michael Parkin

That was one of our initial questions, and I think it’s a question you get a lot, “What is the hierarchical structure?” Because you can go down to the department level of a university, say, and get really granular. For us in EMBL, each site has its own ROR ID, which is very beneficial, but they’re also written fairly similarly. They all have the words “European Molecular Biology Laboratory” in the name, and then you’re looking for basically which city it’s in. So that was the specific project: building this machine learning model that can look at affiliations in articles and pull out the ones that are from our wider organization, and then we can track things like collaborations with other countries. That was a key part of what the strategic team here wanted to do. That wasn’t my work at all, and I don’t take any credit.

The project I’ve been working on more recently was in our grants world. Moving away from the publications, where there are forty-something million records, to the grants world, where it’s much smaller: there are about 70 to 80 thousand grants. It’s a lot more manageable. And so what we’ve been doing is attaching ROR IDs to the institutions that were funded by grants from the Europe PMC funders. Our grant data looks not too dissimilar from a journal article, really. It’s got a title. It’s got an abstract. It’s got a person associated with it, but instead of being an author, we call them PIs, principal investigators, and they are attached to an affiliation of some type. Usually it’s the university, but it can be companies, or it can be charities. I recently saw one that was a cathedral. It’s a real spread of places these PIs can be at.

Amanda French

Yes! I happened to see the ROR ID for York Minster the other day, which is the enormous cathedral in York.

Michael Parkin

That’s a lovely cathedral, actually.

So there’s a real spread of organizations, and the data comes to us through spreadsheets. Some of our funders are relatively small organizations, perhaps with a small number of staff, and anything overly technical is not going to be fit for purpose. We have a spreadsheet where they can fill out all the grant details, and one of the columns of the spreadsheet is the organization that the PI is at, and over the last decade and a half, there’s been a lot of variation in the way that those affiliations get listed. We’ve had the University of Cambridge spelled in about eight different ways, with those including lowercase, uppercase, commas in different places. On our system, if you type “Cambridge,” into the auto-complete menu, you get six or seven variations of “Cambridge University,” so “the University of Cambridge,” “Cambridge, comma, University.” It’s terrible.

So what we were looking for is a way to normalize those. This is probably a very classic use case of ROR here. We’ve wanted to do this for a very long time, actually. This is not a new problem. It gets increasingly worse as time goes by and funders find new ways to write different names for universities and such. We wanted to combine these all, and attaching these affiliations to a ROR ID is a really, really neat way of doing it. On the one hand we get a PID, which is wonderful, something we can share with others and disseminate. But also you have a very well curated registry, which happens to provide a suitable single affiliation name. I think it’s called the primary name in ROR, but we call it internally the “official” name. And, as you know, the ROR registry gets updated. I imagine if some university decides “Actually, I prefer if you use the Spanish spelling of our name,” that would be updated, and we’d incorporate that into our system.

What we’ve been working on is taking our very, very long list of institution names in Europe PMC grant data, many of which are copies of the same institute, just spelled differently, and matching them to ROR IDs. We use the ROR API to do that. We have our machine learning model for publications, but the grant data is a little different. It’s a little bit more straightforward, actually, because these institution text strings tend to just be the institution name, whereas in publications you get postcodes, countries, cities… Sometimes you get two affiliations merged together in the same bit of text. Luckily for us, with Europe PMC grant data, it’s usually just something as simple as “the University of Cambridge”.

So we basically took all institution names, ran them through the ROR API, and had a threshold for matching. The ROR API, beautifully, gives you a score. It tells you how it matched, what kind of similarity was used. We came up with a threshold where “Anything below this, we’re just going to leave this.” We never had the intention that we’d get 100% coverage, because although you do have York Minster, for example, in the ROR registry, we didn’t expect everything to be there. I was actually very surprised at how many companies appeared in the ROR registry. I wasn’t expecting that, but there are many companies that do research and receive external funding.

Amanda French

I was surprised by that as well when I first came to ROR. I was like, “Wow, there are a lot of companies,” you know, and I’d see one and say, “I’ve never heard of this company,” and then I’d Google them and discover “Oh, yes, they do actually do quite a bit of research.” Especially pharmaceutical research, biomedical research. There are lots of small companies, as well as large companies.

Michael Parkin

We knew there’d be some things that we would never expect to be in ROR, so we never went in with an idea that we need to make sure that every single institution we have has a ROR ID associated with it. That just seemed wildly optimistic. And so as we went through this process, we did check everything manually. That was kind of inevitable, that we’d need to do some sort of manual step.

Amanda French

And what kind of volume are we talking about here?

Michael Parkin

I think there’s about 4,000 ROR IDs that we added, so that’s quite a bit. A reasonable amount of checking. I did it over several days to not burn my brain out. But we prioritized institutions that came up a lot in our grants data, and I think I’m right that the top institution was the University of Oxford. That was the most frequently occurring institution I think that was linked to several hundred grants, something like four or five hundred grants just from the University of Oxford, so actually a reasonable percentage of the total. We went from the top down, looking at how many grants are associated with each institution and prioritizing assigning a ROR ID to the top ones, because there are an awful lot of institutions that just appear once. We tried to get as much coverage as we could for as little time investment as possible. But over time we will revisit this kind of exercise. This isn’t an automated process. It was a kind of one-off task. We’ll revisit it, and try to bolster our ROR coverage.

But what we’re hopeful for, and I’m even more hopeful on the publication side of things, is to get funders to provide us with the ROR IDs. One funder does, which is wonderful, actually, that they go to the trouble on their end of figuring out the ROR IDs for these institutions that they’re funding, and then they pass those to us. But we don’t make that a requirement, because as I mentioned, some of these funders are small, and that’s a large burden. We really want their grant data, so we don’t require the ROR ID. In fact, we don’t mandate a lot of grant metadata, to try and make it as simple as possible.

Amanda French

We recommend that people don’t require ROR IDs actually, because sometimes there are organizations which are entirely legitimate for your use case, but are out of scope for ROR, or there are organizations that are in scope for ROR, but happen not to be in it yet. And while we do have a quite robust update mechanism right now, it still can be weeks before something gets in which is not always congruent with people’s timelines for what they need to do.

Michael Parkin

We ask for ORCIDs as well. They are relatively well used, I think. At least one of our funders mandates ORCIDs for their funded researchers.

Amanda French

It makes more sense to me to mandate ORCIDs, because it’s really not that difficult for an individual to just go sign up for one, whereas because ROR is curated, you’re reliant on somebody else to create that for you.

Michael Parkin

Yes. It would be nice if all the funders were providing this data to us at the point the grant enters our system. Then we’d never have to do this process. We’d just take those ROR IDs and add them to the system. But that’s not practical, so this was kind of a jump start. We went from around 10% of institutions in our dataset having a ROR ID, to about 80%. That’s a nice boost.

And as I mentioned, we can disseminate these ROR IDs, because we’ve added them to our own API. When you look at a grant record programmatically, you’ll see the ROR ID as part of that API response. And as you’ve mentioned in your talks, they go into the Wellcome grant DOI metadata as well, so that’s another nice way of making this available to others. I’m a data scientist, so I love that. The more data the better. It’s great to have that kind of information so that people like me can come along afterwards and do some cool stuff that you wouldn’t be able to do otherwise, or that would certainly be a lot more arduous when you have to sift through eight different variations of “the University of Cambridge” and somehow figure out that those are all the same.

Amanda French

And do that for 4,000 institutions.

Michael Parkin

It’s a punishing task for everyone to do that themselves. It makes much more sense for an organization to take responsibility for doing that, and then you can share those results with everyone. So that’s the grants integration. That’s quite different from the publications one, but the latter is ultimately where we want to be, because that’s the main data at Europe PMC – journal articles and preprints and other associated literature. I’m just really hoping that although maybe publishers don’t benefit hugely from putting ROR IDs in their journal workflows, it is extremely useful for everyone else who comes afterwards, including our team, but also for funders, for instance, it would be immensely useful. I know there are a few publishers already doing this. I think eLife and Hindawi at least.

Amanda French

Yep.

Michael Parkin

Europe PMC gets a lot of full-text XML files from PubMed Central. I’d love to see ROR IDs captured in those full-text documents, because then we can reliably start pulling them out. Both of these sorts of initiatives I’ve described are one-off tasks in which we were manually adding a ROR ID to something that already existed. I’d really love to see these pipelines automatically pulling these PIDs in. But it relies on the publisher or the funder to do the work. Because it is work, adding these ROR IDs to your system.

Amanda French

Absolutely. And I think there’s a lot of momentum in that regard. I think if we talk a year from now, there will be a much greater percentage of those publications that have ROR in their data. We know of a lot of service providers who are adding ROR and publishers who are adding ROR. And we would love to see PubMed’s systems officially support ROR.

Michael Parkin

PubMed has supported ORCIDs for quite a few years now. For now, on the ROR community calls, it always cheers me up to see that bump in ROR IDs in Crossref metadata: “Oh, that’s the Wellcome grant DOIs we registered.” Although I enjoy that, I really hope that in a couple of years that’s a tiny little blip because publishers have put so many ROR IDs in Crossref journal metadata that you can’t even see that on the graph. That would be really nice. I’ll be a little bit sad, because once upon a time, you know, Europe PMC had a reasonable contribution to adding ROR IDs to Crossref. But I really do hope in time that you can’t even see it.

Amanda French

Yes, in early August 2022, Europe PMC made a very large spike in the DOIs with ROR IDs in Crossref.

Michael Parkin

Yes, and it’s basically as a direct result of this kind of matching process that we did, so for now it’s substantial. I really hope that publishers will embrace this and start putting ROR IDs in Crossref. And again, then people like me can do some really cool stuff.

Amanda French

What do you hope ROR does in the future? What problems should we fix? What opportunities should we pursue?

Michael Parkin

Phew. For the problem it fixes, as it is now it’s a very good system.

Amanda French

Bug fixes? Feature requests? Suggestions?

Michael Parkin

I mean, the only issues that I had, have kind of been resolved already. They just arrived a little too late for us. The way we get ROR information into our system is to use those data dumps in Zenodo. What we do is we download the JSON file and we parse it out. We’re actually only interested in a very small bit of information: all we want is the ROR ID and what the ROR registry considers the primary name. There’s a lot of stuff in ROR that’s really interesting. In the future I’d love to do more with the data that’s in there. But for now our use case is fairly limited. We would ideally not have to work with the full JSON dump, but now I think there’s a new CSV copy which we’ll probably switch to because we just need to take two columns. With JSON, we use this command line tool called “jq,” and it’s a little bit more involved. But unfortunately the CSV file didn’t exist when we were starting all of this.

Amanda French

We just released that in March.

Michael Parkin

So that was not a problem, but that was a minor inconvenience that you’ve now sorted. And then the other thing would be around improvements to the matching API, but I think that has also been done. At the time the documentation was not so comprehensive. Now I think it is. It’s very clear what to do and the output you’re going to get. At the time I was doing this a year and a half ago it was a bit less clear, but that’s been improved, too. So in terms of the user experience, it’s very good.

Longer term… I mean, ORCID is interesting because you have the PID, but you also have the ORCID record, and I can claim various things to it. The ROR registry does not have the claiming aspect. I wondered if that was a desirable thing or not, because in my head, ORCID and ROR solve the same problem for a different type of thing. But the ORCID organisation has all this flexibility around claiming various bits and pieces to your ORCID record. That proves to be really useful when you want to make connections between things, and we use ORCID as a source of figuring out which of EMBL researchers have published, what grants do they hold, that kind of thing. We actually use their ORCID record to create staff profiles, so you get the little headshot and the list of research publications is just pulled out of the ORCID record for that person. That doesn’t exist for the institution level. The institution doesn’t have the ability to add details to that ROR record. I suspect that is just not in the genetics of ROR. But I wondered if that’s something that there’s interest in. There’s a whole world of complexity, though.

Amanda French

It’s certainly not something that’s on our roadmap now. But what’s funny is that we do get a lot of people sort of assuming that that’s the way ROR works, that we are some kind of membership organization or that as an organization you need to register with us in order to update your record.

Michael Parkin

That you need an account, otherwise it won’t appear.

Amanda French

Right, and none of that is true. Anyone can submit a form and request a change to any record. I’ve certainly done that with organizations I used to be affiliated with, and I said, “Oh, you know, this information should include X, Y, and Z.” But we do actually give a bit of preference to people who are from a particular organization. It’s fairly informal: when we turn things into GitHub issues, we tag it, if it’s a person from that organization who wants to make a change to that organization record. And of course that’s very common. It is usually people’s own organizations that they want to update.

With 105,000 records and more, we would have to have a lot more manpower, I think, to manage 105,000 accounts. But it is interesting. I sometimes wonder what we’d do if we had an edit war between requesters over information in a record. As far as I know, we’ve never had that. We have a curation team that evaluates everything on the merits. Is this a good change? Is this accurate? And we get good advice from others, as well, and the curation team makes the ultimate decisions.

But sometimes I wonder what we’d do if there were competing authorities suggesting what the official name of an organization should be, for instance. We are doing a lot of that, you know. You mentioned as an example earlier changing the primary name in the name element in a ROR record to the name in another language. And we’ve been doing a lot of that, because in the data we inherited, most of those names were in English, and so we get a lot of “Hey, we are a Spanish university, can we have our primary name in Spanish?” And we’re happy to do that.

Michael Parkin

I’d say in general I don’t have any big feature requests in terms of new things to do. I’m a bit at a loss, except to keep doing curation. There must be a lot of work behind the scenes that probably no one sees. But yes, with over 100,000 records that must be a lot of work. It’s very important, I think, that people trust that you’re a reliable source of information. I think that with a lot of automated systems, there’s a lot of drift towards things not being quite right or introducing errors. Having a team of humans that can tackle these requests is a really valuable thing. But it’s a lot of time, and it may be something to consider as the usage gets higher and higher, especially if big publishers start making use of ROR, whether that’s going to lead to an increased amount of curation required. And how do you cope with the increase? Because you can automate processes, but then I think you lose a bit of accuracy in the process, or even just a little less of a handle on what exactly is going on.

Amanda French

Right.

Michael Parkin

And the other thing would be the documentation, and the way that you’re always inviting the community to provide feedback on things like schema changes, that’s excellent, partly because it lets people know what’s coming up, and lets you object if you have a strong opinion about that: “Oh no, that doesn’t work for us, have you considered that this might mess up this kind of system.” I just think it’s a really nice way of doing things. But it is very hard to do and do well, and I think ROR does it very well.

Amanda French

That’s wonderful to hear. We rely on it, and it really is enormously helpful if people can even just skim something and bring us their expertise.

Michael Parkin

Yeah, there are a lot of experts out there who might just think of something you haven’t thought of before.

Amanda French

We find that with every feedback round. “Oh, we thought we had a good proposal. But oh, here are the things that we didn’t think about.”

Michael Parkin

And people can ignore it if they’re not interested.

Amanda French

It’s interesting what you said about curation, because we are absolutely seeing that. The rate of curation requests has been going up and up and up. We really have only been doing this kind of curation for about a year and a half, actually, because before that ROR was just synced with GRID. So yeah, we are right now looking at ways to maintain the pace of that curation in the face of an increased number of requests. And I am also always trying to make that a little bit more visible, to show all of the requests that we do get and all of the curation work that does go on. Because it is open and transparent, but it’s on GitHub, where not everybody goes. It’s trying to really show people where that is and what’s happening.

Michael Parkin

But yeah, the volume of work increasing does pose an issue. Scaling has some challenges.

Amanda French

Yes, and we are figuring out what to do about that.

Michael Parkin

I take it that ROR is completely diverged from GRID at this point, so you’re entirely responsible for the data, and there are no updates from GRID to feed into ROR? At this point it’s always a request directly to ROR?

Amanda French

Yes, that’s correct.

Michael Parkin

And presumably no plans to remain in sync?

Amanda French

Well, no, none at all. GRID no longer has public, resolvable IDs. The IDs do exist, but they are using them really only internally in Digital Science products like Dimensions. There are no public updates to GRID at all anymore.

Michael Parkin

I see.

Amanda French

Well, that’s about it from me. Anything else you want to say?

Michael Parkin

No, I’ll let you go.

Amanda French

Thank you so much for you being such a big supporter of ROR in general, and for giving us feedback and coming to our events.

Michael Parkin

My pleasure. Thanks for your time.

Amanda French

Thank you so much for agreeing to interview with us. Can you start by telling us your name, title, and organization?

Andy Dobson

I’m Andy Dobson. I’m the Client Services Director at DataSalon.

Amanda French

Tell us about DataSalon.

Andy Dobson

DataSalon was originally set up in 2006 by Nick Andrews, who is our Managing Director, and Jon Monday, our Technical Director. They had been working together managing large-scale IT projects for major publishers and retailers, and had first-hand knowledge of the technical systems and services used by publishers. DataSalon was born out of the challenges, needs, and requirements faced by the publishers they were working with.

Our original product is MasterVision, and that’s really core to everything that we do here. Not only was it the product that gave the business its start originally, but it also became the basis for a lot of the services we provide, such as PaperStack, and the data quality work we do around cleaning and standardizing data. MasterVision was developed in response to the problem of establishing a joined-up view of multiple databases without having to replace or upgrade existing systems. From its launch the system was immediately adopted by OUP and the BMJ and was acclaimed as solving a significant business challenge for a fraction of the time and cost of conventional solutions. I’m pleased to say that both OUP and the BMJ are still clients!

We’re quite a small company, but we’ve got a good long-term client base at DataSalon. People like the way we work, and they continue to work with us. We have a very collaborative relationship with our clients. We help them solve a lot of the challenges around data cleansing, integration, and analysis of their data in a way that’s significantly faster and cheaper and more flexible than conventional approaches. And as an organization we’re quite small, but we’re extremely agile, so we’re very responsive in terms of customer service, and we’ve got an excellent understanding of the scholarly publishing industry.

I’ve previously worked at Elsevier and Ingenta, and all of the staff have worked at various companies within scholarly publishing at some point. So although we’re an IT company, we’ve very much got one foot inside scholarly publishing.

Amanda French

So you’re saying that you’re small but mighty?

Andy Dobson

Yes, small but mighty. Exactly. We like to think of ourselves as being a bit different, innovative, and extremely effective. We’re a 4-day week employer, and we’ve always worked, in the main, from home. Even before the COVID days, staff have generally worked from home, so we’ve been quite forward-thinking in that sense.

The key to our success has been in ‘growing’ each client’s system directly from the source data, rather than creating them through the traditional time-consuming (and expensive) process of analysis - design - implementation. This same innovative principle lies behind all of the services we now offer, including PaperStack. We offer a very collaborative way of working with the data that is customized to the needs of the publisher. So right from the setup of a given build of MasterVision or PaperStack we work with our clients to develop the tools alongside the specific requirements of their data.

There are a lot of similarities, obviously, across scholarly publishing data. Typically it’s subscriptions data, memberships, usage, denials, etc. – all of those things that we’re familiar with. But there are, within that, very individual requirements related to their business models and company goals, so we work with each one individually to give them what they need.

Amanda French

Great. Can you tell us a little bit about first MasterVision and then PaperStack? What kinds of things do they provide for publishers?

Andy Dobson

MasterVision was our first product and it provides publishers with complete customer insight via a fast and user-friendly hosted service. It painlessly merges all of the valuable data from different source systems providing a ‘single customer view’ of all interactions with a customer. The interface itself is a standard across all of our clients, but each one has been tailored to their individual requirements. There wasn’t anything doing that for publisher data at the time MasterVision was created, and I still don’t think there is anything that offers that in quite the same way.

We don’t look to replace any of the source systems. Publishers will still pick and choose the most effective systems that they want for fulfilling their needs, ie. their platform hosts, their membership database, fulfillment, and peer review systems etc. The integrity of that data is kept and what we’re doing is providing a read-only interface onto all of that data. So we’re effectively joining up all the different instances of the given customer, whether that be an individual or institution, into a single record, and maintaining and describing that customer hierarchy for detailed analysis. It comes in really useful if you want to analyze things like “How many authors have I got submitting to our journal from a given institution?”

Amanda French

I can see why you would be interested in ROR, then.

Andy Dobson

Yes, exactly. So having a reference record to tie customers to is really important, and having the hierarchy of those customers offers vital context. MasterVision is predominantly a sales and marketing tool, and allows publishers to look for sales opportunities and to provide better customer service through having a deep understanding of their customers. If you’ve got customer data in multiple datasets living in multiple systems, it’s time-consuming to go in and query each of those separately. Whereas pulling all those together into a system, that can’t be broken by end users, allows staff to see exactly what’s going on on a customer basis for all aspects of the relationship. And it’s extremely quick and simple to get answers. So rather than getting IT involved to run complex queries across multiple systems, the end user, even if they’re non-technical, can do that themselves. You can put together quite complex queries with MasterVision: “Who does XYZ, but doesn’t do A or B?”

Amanda French

It sounds like you anticipated data science before “data science” was really a term.

Andy Dobson

Yes, we’ve been doing this for quite a long time now. We feel like we were one of the first to really promote data quality and data cleansing. And as I say, many clients stay with us for a long time.

Our other product, PaperStack, works on a similar basis. It’s providing publishers with a complete reporting suite for the scholarly submissions process. It is fully integrated with major peer review systems including ScholarOne and Aries Editorial Manager, meaning there is almost zero publisher overhead to get up and running. As a standard part of the service, data from peer review systems is also automatically linked and enhanced using industry data from ROR or Ringgold, Funder Registry, ORCID, and Crossref - all helping to maximize reporting insights. This results in a clean, standardized and comprehensive cross-journal view, available at the institutional, funder, author, and article level. As far as possible PaperStack aims to provide a service with zero overhead in terms of effort on the publisher’s part. What we’re doing is providing a contextualized and enhanced view of their peer reviewed data, including details of where rejected articles end up.

By “contextualized,” I mean that we’re using ROR, Ringgold, Funder Registry data, ORCIDs, publication information from Crossref, etc., for building out a comprehensive view of not only the timeline of a given article, but also all of the other big pieces of the puzzle: What are the affiliations of the authors and the editors? What is the hierarchy in terms of organizations they’re affiliated with now? How was that then and what does it look like now? How long did it take an article to be accepted? Where did it ultimately get published? We use all that data to create a set of standardized reports that publishers can use, as well as helping them to create additional customized reports that they might want to design to meet their business needs. So PaperStack provides extremely detailed, responsive, and visually appealing reporting of their scholarly submissions.

Amanda French

And do you think that’s also primarily used by sales and marketing teams at publishers? Or does it have a different purpose?

Andy Dobson

It’s primarily editorial staff that use PaperStack, but there are applications for sales and marketing in terms of some of the reports that could be created. Additionally you could add in any additional customer data to enhance the overall picture. Because it’s based on MasterVision, it potentially means clients could load in usage and denials, APC data etc. to add further value to the submissions picture.

Amanda French

Great! I think I’ve got a sense of what they do now. Tell me, who were the primary advocates of implementing ROR at your organization?

Andy Dobson

I would say all of us, from the customer service side to the product managers, to the technical team, and Nick himself. One of our product managers managed the implementation, but the requirements from our clients meant that there was a lot of support and interest from all parts of the business, really, in getting it within the products that we use. We’re really interested in data standardization and cleansing: that’s central to what we do, so we’re all very aware of the importance of having a free and open standard reference dataset for identifying organizations and for supporting some of the other datasets that get loaded into our products. Open source data is key to our ethos.

Amanda French

That’s wonderful. You’ve mentioned the importance of a free and open dataset, and your products do use Ringgold as well, as you’ve mentioned. Can you talk a little bit about what ROR does that Ringgold doesn’t? And vice versa, if you like, too: what does Ringgold do that ROR doesn’t?

Andy Dobson

Yes. Just to go back slightly: as an organization, we’ve always used an org dataset for standardizing and linking academic institutions within MasterVision and PaperStack. It’s key for what we do. And while that’s not completely necessary for the products themselves, it does provide an underpinning for delivering a comprehensive and reliable single customer view. In the past we’ve always used Ringgold for those clients that license the Ringgold data. But there are a number of clients for whom the Ringgold dataset isn’t affordable, practical, or streamlined enough for their purposes. Our client base is pretty much half and half, I would say, between the publishers that used Ringgold and the people that didn’t have anything. Back in 2014 we felt that there was a big need for us and our clients to have access to a free and open dataset. I don’t know how familiar you are with OrgRef, but we launched OrgRef in 2014.

Amanda French

I’m somewhat familiar with it.

Andy Dobson

Back in 2014 we launched OrgRef as the first free and open dataset about academic institutions. We worked mainly to establish some useful metadata around academic institutions and a hierarchy picture to provide a way of standardizing that information and making it available to the wider community. I think when OrgRef was launched there wasn’t any other free and open institutional ID available. We gave it away for free, and we used it within our products. We received a lot of useful input, from our clients, to develop the dataset further.

It was well received, but obviously the resource that we could attach to it was a lot smaller than commercial offerings, so there was a sizable gulf in terms of the volume of records that sat within the dataset. Ringgold had about 400,000 or so at the time, and we had I think about 80,000 – although those 80,000 pretty much covered all of the likely subscribers to academic content. Anything more than that tends to include orgs a lot lower down the institutional hierarchy. Not all scholarly publishers are going to be interested in seeing a secondary school in the middle of California that’s not likely to buy a subscription. There is a lot of data within Ringgold, for example, that won’t be necessary for the needs of the smaller publishers.

Amanda French

We’ve definitely heard that before, and of course we appreciate it. ROR currently has about 105,000 records.

Andy Dobson

OrgRef itself was extracted from structured information on Wikipedia and other open resources. And, as I say, it aimed to cover most of the important academic and research organizations. The metadata was similar to what you would see in Ringgold – things like name, country, URL, standard IDs like VIAF. We made OrgRef free and open for people to download. It was maintained by us and updated on a monthly basis. That probably was less frequent than some publishers would like, in the sense that they might be wanting to update their customer data on a more regular basis. And so we managed that for a while, and then Digital Science launched their GRID dataset, and they had more resource available to put into that. Their metadata was slightly richer. So we assisted GRID, and they used the OrgRef data to enrich the GRID dataset.

Amanda French

That was very generous of you.

Andy Dobson

Well, it was free and open anyway! But we stepped back and let them carry on, just because we’re quite a small company. We had other products and projects that we were more focused on. The main goal was really to have an open data set that could be used by those who couldn’t afford Ringgold and that could be used within our products. If someone else was willing to take that on, that was fine by us.

Amanda French

They were very helpful in setting up ROR.

Andy Dobson

Yes. Although it’s great to do it as an open resource, I suspect both Digital Science and ourselves partly did it for selfish reasons as well as altruistic reasons. So I think when it gets to the point that someone else is prepared to take over, you say, “Yeah, that’s great, thanks.”

Amanda French

Right.

Andy Dobson

And ROR probably sits within that Crossref space better. A lot of what we were doing was pretty much automated, although there was some manual input. As I understand it, there’s more manual input on the Ringgold side for digging out the right record, making sure that that is the right address, etc. They’ll resolve institutional lists that you might want to provide to them, and they do some other bits and pieces that sit outside of what is possible with automation alone. It comes down to, I suppose, publishers getting what they pay for. But having a free and open reference dataset is great for everyone.

Amanda French

Great. Can you take us through some details of what you’ve done in your ROR integration? How did that process go? What does that look like within MasterVision in particular, but also PaperStack?

Andy Dobson

Yes, okay. The ROR integration was very straightforward. We use ROR in both our core services, MasterVision and PaperStack – both of these join together data from multiple sources to give a complete picture of customer activity (for the former) and the editorial process (for the latter), and to do that we need to identify which records relate to the same institution or individual. Matching with ROR means we can link up all the activity by a particular customer even if the data doesn’t have any other IDs or if multiple customer IDs have mistakenly been assigned to the same institution – which happens a lot!

It also means we can standardize the data – we use the name and location information from ROR for the main display, so that the format is consistent across all data sources. We do the matching not just for institutions but also for individuals, to link them to their associated institution, which means that our clients are able to analyze data available at the individual level (like article submissions or pay per view purchases), as well as at the institutional level.

We also use the parent/child information included in ROR to show the relationships between records, with a family-tree style hierarchy viewer. Making use of hierarchy data allows clients to achieve complex uses around activity, be it funding, article submissions, content purchasing at any level in that hierarchy, or for example, do institutions access content that their associated authors submit to.

Amanda French

And how did you match the ROR IDs to the existing institutional data?

Andy Dobson

It was quite simple to do because ROR data is very much similar to GRID. We would just use ROR in the same way as the GRID data that we’d previously been loading. Because ROR contained exactly the same data as GRID, including GRID IDs, it made it easy for us to test and implement. We migrated each client site, and we were able to check that the matching tools that we used were always giving the same results as ROR IDs. There’s lots of challenges around that, particularly using real-life data. Using a reference dataset means that we can standardize information within the real data from publishers, which is pretty messy most of the time and not consistent from system to system. There’s things like different name forms, different location information, different data structures, data entry errors, missing location information. But a lot of those things we’d already addressed and developed within our own matching tools.

Amanda French

Gotcha.

Andy Dobson

Part of what we do is provide matching services to publishers as well, so that sits within the tools that we have. As we load data, we will match to the reference dataset where we can. Where there are IDs that we can match, we will use those. If not, we will use our own tools to auto-match against that reference dataset. So we’re using that dataset in two ways. One is ‘as we find them’, matching customer IDs that might sit within the publisher data to the reference dataset, and the other is to match the metadata to the reference dataset.

Amanda French

And when you say customer IDs, what do you mean?

Andy Dobson

If the fulfillment data, for instance, that we receive from a publisher contains a ROR ID, or a Ringgold ID, we’ve immediately got a match to the full ROR or Ringgold record. So where those are used in the publisher data, we will match that to the standard records.

Amanda French

Gotcha.

Andy Dobson

So for example, in their fulfillment data, they might have a customer specific ID. Whereas in their usage and denials data they might have a ROR ID which identifies a given org. Where we can pick out those IDs, we’ll use them, but we’ve got our own matching tools that we’ve developed over the last 15+ years that allows us to do automated matching to create a link to the ROR data, where those IDs don’t exist. The benefit is that all those different instances of “University of Oxford” will match to one “University of Oxford” record.

Amanda French

Right!

Andy Dobson

We’re doing work on our side to match to those standard records. We use a variety of strategies like fuzzy matching; lists of alternative names to supplement the ones in ROR; lists of alternative names to ignore (so that common acronyms shared by more than one organization don’t lead to incorrect matches); lists of synonyms for typos and US/UK spellings; mappings of state and country codes to full text; inferring missing information from the country name; and matching email and web domains against the organization URLs in ROR.

Amanda French

Wow.

Andy Dobson

We are very proud of our auto-matching tools which can link ‘real world’ (i.e., messy!) publisher data to the correct ROR records with an impressive degree of accuracy. The rules and exceptions in play there have been refined over many years. For example, ROR ID https://ror.org/04cvxnb49 (Goethe University Frankfurt) brings together many different names:

• Goethe University
• Goethe University, Frankfurt
• Goethe-Universität Frankfurt am Main
• Goethe-University
• Johann Wolfgang Goethe University

Amanda French

It’s fascinating, and you do have a a nice blog post detailing some of what you’ve you just told me – your “secret sauce.” We’ve found that that matching is a thing that many people and organizations want to do, because there are a number of organizations that, as you say, had nothing, and so they have a pile of messy text strings, and we do have tools and methods that we recommend for doing that. But it’s been remarkable to me in the last few months to discover how many different methods people are using. A lot of people are building their own systems for doing that, and they don’t have your fifteen years of experience doing that matching with other things.

Andy Dobson

What we’ve got in place is very well tried and tested now and works really well in real-life scenarios. There can be multiple different versions of Oxford University that might sit within the different datasets that we pull in, whether it’s “University of Oxford,” “Oxford University,” or indeed a subunit that sits below that top level, something like the Bodleian Library, which would be considered as part of the University of Oxford.

Amanda French

Is the matching that you do entirely automated, or is there any human checking afterward?

Andy Dobson

There is some manual checking that goes on, but it’s predominantly automated. I would say probably 90% is automated. There is a manual checking process where we’ll go through and make sure there’s no obvious incorrect links there. That gets used in a couple of different ways. The first is on the fly, during the build processes for our two products, MasterVision and PaperStack. What we’re doing is we’re taking weekly data uploads from the publisher and then rebuilding those sites for them on a weekly basis, sometimes on a daily basis. Part of that build process will be running that matching against the latest data that we receive so that the publisher has the latest, best, single standard record for a given institutional customer based on the latest data available. In that sense, it’s working in an automated way as part of the build process, so if there’s half a dozen versions of “Oxford University,” then we would link those up during that build. Obviously, if there are IDs that we can use from Ringgold or ROR, then we can use those in the build process for creating that single standard record.

And then we also use the auto-matching tools that we have to provide additional services to publishers. Some might not be MasterVision or PaperStack clients. They might come to us and say, “We’ve got 10,000 institutional records that are just a complete mess from this particular database that we’ve been keeping for the last ten years. Can you deduplicate it? Standardize it? Match it to ROR?” And we can do that for them. It’s part of the service that we can provide as well.

Amanda French

Oh, that’s interesting! I didn’t know that. As I said, we actually have been hearing a lot of desire for that from different entities.

Andy Dobson

Yes, it’s a pretty straightforward process. By its nature it’s not 100% accurate, because often the data is so messy and what’s available just isn’t clear enough that some matching is just not possible. Oftentimes the data might have been collected over a period of years from lots of different places. There might be someone who has been to an industry show that’s just taken some notes down and has gone, “Well, you know, X university has got this address,” and then it sits within the database and no one knows what to do with it or how clean it is. So, yes, we often do projects for people where they want matching services to things like ROR or Ringgold if they license that.

Amanda French

You’ve said that the matching is not 100% accurate and that 90% of the matching is done automatically. Do you have any rough metrics that you know or can share about how accurate your matching is if you just do it automatically?

Andy Dobson

That’s a good question.

Amanda French

I know it’s hard to tell.

Andy Dobson

It’s tricky, because it depends on what the original data is like. Sometimes the matching might be 90%, sometimes it might be 70%; it just depends on how clean the input data is. But I would say around 80% is probably right.

Amanda French

And then, whatever the gap is, you’ll check that manually?

Andy Dobson

Well, it depends on what the client wants us to do. If they want us to do some manual work to try and fill in some of the gaps, we can do that. However, oftentimes an automated job matching a large chunk of data that otherwise couldn’t be used is good enough. Also, in some cases, if the data is really messy and sparse, and there is really not much evidence to allow us to make a match, then even with some manual input it would be difficult to have a match that you were 100% sure was the right one.

Amanda French

Right, because you just can’t tell what people mean. What do they mean?

Andy Dobson

Well, there are two big “York University’s” in the world. There’s one in Canada and one in the UK. There might be another one somewhere as well. If someone’s put “York University” and there’s little additional contextual information, you wouldn’t be 100% sure on which one they meant. So in some cases there’s some ambiguity there even when you get down to a manual level, because you just wouldn’t be able to tell which was which. There is an element of not having all of the information to allow you to be 100% accurate. We could literally be receiving a file from someone coming to us and saying “We found this file lying around on an old system. It has 10,000 names, and we don’t really know what to do with it. Can you do something?” So the starting point can often be challenging.

Amanda French

Got it. Let me ask you, too, how do you get the newest ROR data? Are you using the ROR API? Or do you get the ROR data dump?

Andy Dobson

We get the ROR data dump. We do use APIs for our PaperStack service when it comes to actually loading certain information. We use the ScholarOne APIs and the Editorial Manager API for collecting the data. However, we use the ROR data dump itself for loading into our services.

Amanda French

We are doing releases now at least once a month, usually twice a month. Do you have a programmatic way of getting those new updates to ROR, or are you just redownloading it from Zenodo every time?

Andy Dobson

We download it on a monthly basis, so whenever there’s a new one that comes out, we will pick that up and then apply it to the data that we’ve got.

Amanda French

Perfect. Just a couple of more questions. What do you hope ROR does in the future? Do you have any bug fixes, feature suggestions, advice, requests?

Andy Dobson

Our main concern previously was that there was no concept of inactive organizations, so it’s great that that’s been addressed. Going forward, I guess the best thing from our point of view would be adding new records just to increase the scope of the dataset.

It’s also key for us to keep the organizational hierarchies lean and relevant. As we talked about, having too many layers just takes away the usefulness of it and muddies the water a bit. You don’t need to know ten layers down what’s going on, because it’s likely not important or not relevant. That’s probably the main thing from our side, really, just for what we do, because that allows us to keep things like hierarchies within MasterVision and PaperStack understandable. If you’re looking for a sales prospect, or you want to understand at what level a subscription’s sitting at, or you want to know what authors are submitting at what institution, you don’t need many levels. You want to know those direct relationships, and maybe one or two layers down, but you don’t need to know that someone worked in a department of a library at the college of X university, which probably isn’t going to be particularly relevant to the sorts of analysis you want to do as a publisher.

Amanda French

What I’m also wondering about is what the business case for ROR is – for you, but also for publishers or other entities. How is ROR adding value to for-profit enterprises, given that it itself is free and open?

Andy Dobson

It comes down to the standardization of your customer data and to have a unique way of identifying a given customer / organization, within the sphere of your relationship and within scholarly publishing more widely. If you can’t associate all of your customer activity to a standardized picture of each organization, then what you’re left with is a very messy picture that is difficult to understand, and it’s very time-consuming to analyze and get into the guts of what your relationship with them ultimately is. If you’ve got multiple versions of an institution that you’re not tying together in a single record, then you don’t know, potentially, the full value that X University is getting from the deal.

For example, if a customer is referenced differently within both COUNTER data and fulfillment data, and you can’t join those records together, then you’re not going to be able to understand how much use they are making of their subscriptions. It also has a big impact on the customer relationship. When it comes to things like renewing sales, if you can’t actually tie up all of the different sublevels of the hierarchy of Oxford University to the top level, you could end up trying to sell things they already buy and it becomes a very untrusting relationship.

Amanda French

Yes indeed.

Andy Dobson

If you’re making a decision as a library based on how much content is being used, then you want to make sure your usage numbers align with what the publisher is seeing. And as a publisher, if you’re not able to present consistent numbers because you’re not able to tie up usage of the Bodleian Library with the University of Oxford, then it becomes a very difficult conversation. It’s like any customer relationship: one needs to feel valued and fully understood. None of us want to be mistaken for somebody else!

Amanda French

Definitely.

Andy Dobson

And if you’re moving into OA, and you’re not understanding a standardized picture of your customer hierarchy, and you don’t know how many articles are coming out of the University of Oxford and how much they should be paying in article fees, then that becomes really tricky. There are potentially significant financial implications, on the customer service side, of not having a dataset that allows you to clean up and standardize all of those customer pictures within your data. Having access to ROR and services like MasterVision allows publishers to continue to maintain and use their existing databases, and the imperfections therein, whilst being confident that they can clean and standardize data in an effective and economical way further downstream.

Amanda French

Right. We hear that a lot, that collecting beautifully structured, accurate metadata at the beginning is the ideal, but it just can’t always happen that way. Particularly in the case of affiliations, which were collected so often just in a text field, it can be very difficult.

Andy Dobson

Yes, exactly. While we would always recommend putting good data governance practices into place, that is not always possible or affordable. Having access to services that allow one to get the most out of what one has is the next best thing.

Amanda French

Yep. That’s about all I have for you. What else do you want to say about ROR or your implementation of ROR?

Andy Dobson

I think I probably covered most of it. ROR does open up new possibilities for us, such as integrating some of the other systems that are around at the moment. We’re currently looking at bringing OpenAlex IDs into PaperStack. That should be really straightforward, because you can use the ROR IDs to retrieve the OpenAlex IDs through their API. So that’s one of the benefits of having the ROR IDs and the mappings that we’ve got in place on our side. It’s opening up some other possibilities for us, which is nice. But in the main, having that free and open data is beneficial to everyone!

Amanda French

Great. Well, thank you so much again for doing this with us, Andy.

Andy Dobson

Thank you for the opportunity.

Versión en español sigue abajo

Persistent identifiers are playing a key role in driving more robust research infrastructure and open science initiatives across Latin America. This was a primary theme at the event “Persistent Identifiers (PIDs) and Open Science in Latin America” (#PIDsLATAM23) held on April 18 in Buenos Aires (Argentina) during csv,conf,v7.

Organized by DataCite, ORCID, and ROR, the event was attended by more than 70 research stakeholders from across the Latin American region and elsewhere, representing 40 different institutions in total.

Aimed at research administrators, library directors, and technical staff, the full-day program featured a series of in-depth presentations—all in Spanish—about how persistent identifiers are being implemented and used in national, consortial, and institutional settings to advance open science and increase the discoverability and visibility of research in Latin America. The goal was to present use cases and success stories and bring together stakeholders with shared interests and challenges in a Spanish-language setting.

The first set of presentations focused on large-scale initiatives that are aimed at enabling open science with persistent identifiers. Gustavo Durand from The Dataverse Project discussed how PIDs are being implemented in the platform to make research data and other works more openly available; Abel del Carpio from CONCYTEC in Peru discussed the organization’s strategy with regard to leveraging PIDs on a national level; and Washington Segundo from IBICT and LA Referencia discussed efforts in Brazil and across Latin America to develop open science infrastructure using PIDs.

Review the Session 1 presentations

• Gustavo Durand, “Dataverse y los PIDs,” https://doi.org/10.5281/zenodo.7860265

• Abel del Carpio, “Plataformas de CTI y los identificadores persistentes (PIDs) en el Perú,” https://doi.org/10.5281/zenodo.7860287

• Washington Segundo, “Los PIDs y la ciencia abierta: perspectiva Brasil y América Latina,” https://doi.org/10.5281/zenodo.7860279

The next series of presentations focused on global persistent identifier services for enabling more open and interoperable research infrastructure across Latin America and beyond, featuring DataCite (Gabi Mejias), ORCID (Ana Cardoso), and ROR (Maria Gould).

Review the Session 2 presentations

• Gabi Mejias, “Conectando la investigación con DataCite,” https://doi.org/10.5281/zenodo.7860319

• Ana Cardoso, “PIDs y ORCID,” https://doi.org/10.5281/zenodo.7860315

• Maria Gould, “ROR, El Registro Abierto de Identificadores Persistentes para Organizaciones de Investigación,” https://doi.org/10.5281/zenodo.7860464

The third and final series of presentations focused on specific implementations of persistent identifiers in Latin American research infrastructure. This session highlighted institutional use cases at the Universidad Nacional de Rosario in Argentina (Paola Carolina Bongiovani, Paulina Freán, Analía Salazar) and Universidad de Chile (Rodrigo Donoso) to implement persistent identifiers in library and repository systems, and examples of consortia-level implementations by Consortia from Colombia (Paula Saavedra) and eScire (Nydia Lopez and Joel Torres) from Mexico.

Review the Session 3 presentations

• Paola Carolina Bongiovani, Paulina Freán, Analía Salazar, “Implementación de PIDs en América Latina REPOSITORIO DE DATOS ACADÉMICOS RDA-UNR dataverse.unr.edu.ar,” https://doi.org/10.5281/zenodo.7860470

• Rodrigo Donoso Vegas, “Implementación PIDs en América Latina. Experiencia de la Universidad de Chile,” https://doi.org/10.5281/zenodo.7860501

• Paula Saavedra Ochoa, “Gestión de identificadores persistentes consorciados,” https://doi.org/10.5281/zenodo.7860524

• Nydia Lopez and Joel Torres, “Consorcio eSCIRE,” https://doi.org/10.5281/zenodo.7860513

At the conclusion of the program, audience members participated in an interactive brainstorming session to share their impressions about the state of PIDs implementations in Latin America and raise questions and ideas about how to move forward to achieve greater adoption. The participants responded to three prompts:

• What would you like to see happen with PIDs in Latin America?

• What are the challenges to PIDs adoption in the region?

• What can we do to address these challenges?

In the responses to the online poll, several themes stood out: the importance of shared infrastructure and of collaboration, the necessity of making PIDs infrastructure more accessible, the value of having more training and capacity-building, and the ongoing need to continue raising awareness of how to work with PIDs and of the benefits they provide.

Throughout the day, the sessions and participant engagement demonstrated not only a high level of interest in the topic of PIDs and open science in Latin America, but also generated new ideas and opportunities for future collaborations and follow-up actions.

We at DataCite, ORCID, and ROR express our gratitude for the effusive response to this event and the satellite discussions that took place on the preceding and following days, all of which speak to the leading role that Latin America is playing in developing infrastructures, policies, and practices for promoting collaboration around open knowledge, redistributing global research networks, and making it possible for all countries to have greater access to science and technology. It was great to meet the Latin American community in person and we hope to continue working together to build an open and robust research infrastructure!

PIDs y Ciencia Abierta: Construyendo comunidad en América Latina

Publicado a la vez por DataCite y ORCID

Los identificadores persistentes están jugando un papel clave en impulsar una infraestructura de investigación más sólida e iniciativas de ciencia abierta en América Latina. Este fue el tema principal del evento “Identificadores Persistentes (PIDs) y Ciencia Abierta en América Latina” (#PIDsLATAM23) realizado el 18 de abril en Buenos Aires (Argentina) en el marco de la de csv,conf,v7.

Organizado por DataCite, ORCID y ROR, el evento contó con más de 70 personas vinculadas a la investigación en América Latina y otras regiones, representando más de 40 instituciones diferentes.

Dirigido a gestores de investigación, directores de bibliotecas y personal técnico, el programa de un día completo contó con una serie de presentaciones detalladas, todas en español, sobre cómo se implementan y utilizan los identificadores persistentes en entornos nacionales, consorciados e institucionales que apoyan el acceso abierto, la ciencia e incrementan la descubribilidad y visibilidad de la investigación en América Latina.

El objetivo fue presentar casos de uso e historias de éxito así como reunir a personas con intereses y desafíos en común en un espacio de habla hispana.

El primer grupo de presentaciones se centró en iniciativas a gran escala que tienen como objetivo promover la ciencia abierta a través de identificadores persistentes (PIDs). Gustavo Durand de The Dataverse Project expuso cómo se están implementando los PIDs en la plataforma para hacer que los datos de investigación y otros trabajos estén disponibles de manera más abierta; Abel del Carpio del CONCYTEC habló sobre la estrategia de la organización en el aprovechamiento de los PIDs en Perú; y Washington Segundo de IBICT y LA Referencia mostró los esfuerzos tanto de Brasil como de otras regiones en América Latina para desarrollar una infraestructura de ciencia abierta utilizando PIDs.

Presentaciones de la Sesión 1:

• Gustavo Durand, “Dataverse y los PIDs”, https://doi.org/10.5281/zenodo.7860265

• Abel del Carpio, “Plataformas de CTI y los identificadores persistentes (PIDs) en el Perú,” https://doi.org/10.5281/zenodo.7860287

• Washington Segundo, “Los PIDs y la ciencia abierta: perspectiva Brasil y América Latina,” https://doi.org/10.5281/zenodo.7860279

El siguiente grupo de presentaciones se centró en los servicios de identificadores persistentes globales para construir una infraestructura de investigación más abierta e interoperable en América Latina y a nivel global, con DataCite (Gabi Mejias), ORCID (Ana Cardoso) y ROR (Maria Gould).

Presentaciones de la Sesión 2:

• Gabi Mejias, “Conectando la investigación con DataCite,” https://doi.org/10.5281/zenodo.7860319

• Ana Cardoso, “PIDs y ORCID,” https://doi.org/10.5281/zenodo.7860315

• Maria Gould, “ROR, El Registro Abierto de Identificadores Persistentes para Organizaciones de Investigación,” https://doi.org/10.5281/zenodo.7860464

El tercer y último grupo de presentaciones se enfocó en las implementaciones y desarrollo de identificadores persistentes en la infraestructura de investigación de América Latina. Esta sesión destacó casos de uso institucional en la Universidad Nacional de Rosario en Argentina (Paola Carolina Bongiovani, Paulina Freán, Analía Salazar) y la Universidad de Chile (Rodrigo Donoso) para implementar identificadores persistentes en sistemas de bibliotecas y repositorios. Y por último se destacaron algunos ejemplos de implementaciones a nivel consorcial por Consortia de Colombia (Paula Saavedra) y eScire (Nydia López y Joel Torres) de México.

Presentaciones de la Sesión 3:

• Paola Carolina Bongiovani, Paulina Freán, Analía Salazar, “Implementación de PIDs en América Latina REPOSITORIO DE DATOS ACADÉMICOS RDA-UNR dataverse.unr.edu.ar,” https://doi.org/10.5281/zenodo.7860470

• Rodrigo Donoso Vegas, “Implementación PIDs en América Latina. Experiencia de la Universidad de Chile,” https://doi.org/10.5281/zenodo.7860501

• Paula Saavedra Ochoa, “Gestión de identificadores persistentes consorciados,” https://doi.org/10.5281/zenodo.7860524

• Nydia López y Joel Torres, “Consorcio eSCIRE”, https://doi.org/10.5281/zenodo.7860513

Ya para finalizar la agenda, los miembros de la audiencia participaron en una sesión interactiva de lluvia de ideas (brainstorming) para compartir sus impresiones sobre el estado de las implementaciones de PIDs en América Latina y plantear preguntas e ideas sobre cómo avanzar para lograr una mayor adopción. Los participantes respondieron a tres preguntas:

• ¿Qué te gustaría que sucediera con los PIDs en América Latina?

• ¿Cuáles son los desafíos para la adopción de PIDs en la región?

• ¿Qué crees que podemos hacer para abordar estos desafíos?

En las respuestas a la encuesta en línea, se resaltaron varios temas: la importancia de la infraestructura compartida y de la colaboración, la necesidad de hacer que la infraestructura de PIDs sea más accesible, el valor de tener más capacitación y desarrollo de capacidades, y la necesidad constante de continuar creando conciencia de cómo trabajar con PIDs y de los beneficios que estos brindan.

A lo largo del día, las sesiones y la participación de los asistentes demostraron no solo un gran nivel de interés en el tema de los PIDs y la ciencia abierta en América Latina, sino que también generaron nuevas ideas y oportunidades para futuras colaboraciones y acciones de seguimiento.

Desde DataCite, ORCID y ROR expresamos nuestro agradecimiento por la positiva y efusiva respuesta a este evento y las discusiones satélite que tuvieron lugar en los días anteriores y posteriores, todo lo cual habla del papel de liderazgo que está jugando América Latina en el desarrollo de infraestructuras, políticas y prácticas para promover la colaboración en torno al conocimiento abierto, redistribuir las redes globales de investigación y hacer posible que todos los países tengan un mayor acceso a la ciencia y la tecnología. ¡Fue increíble conocer a la comunidad latinoamericana en persona y esperamos seguir trabajando juntos para construir una infraestructura de investigación abierta y sólida en la región!

Amanda French

Thanks for joining us! Please tell us your name, title, and organization.

Cory Schires

My name is Cory Schires. I am a co-founder and CTO at Scholastica.

Amanda French

Tell us about Scholastica.

Cory Schires

Scholastica was founded in 2012 by myself, Brian Cody, and Robert Walsh. We met at the University of Chicago. To this day, we all share a deep affection for academia. Scholastica now has more products, and we’ve expanded over the last ten years, but our core mission remains the same. We want to empower academic organizations of any size to publish top quality journals, more effectively, efficiently, and affordably. We’re also huge proponents of open access. Currently, all the journals published on Scholastica are open access.

Amanda French

That’s amazing. When and where did you personally first hear about ROR?

Cory Schires

I don’t remember exactly. It was probably in the context of Crossref. Actually, I also often find out about industry developments on Scholastica’s blog, run by our marketing director Danielle Padula. It’s a good resource. Like others in the industry, I’m busy trying to keep track of many different things, so I use our blog to stay abreast of new developments.

Amanda French

I was about to say! Danielle is so impressive. She actually just published an interview with me, so we have dueling interviews. She’s very organized and on top of things. I’m jealous.

Cory Schires

So am I.

Amanda French

Who were the primary advocates of implementing ROR at your organization?

Cory Schires

I would say it was the sales team and Brian Cody, our CEO. The sales team is obviously always talking to a lot of prospective customers and sometimes to current customers, and people are always asking for things. We’re very dutiful about keeping notes on all of that and tracking what people are asking for. So while we heard it from the sales team, they’re ultimately relaying customer requests. If enough people ask for something, we’ll probably build it.

Amanda French

You’re obviously very good about announcing new features in your products. We’ve seen you do that for ROR, and for other features as well. I’ve been reading around in your announcements, and they are frequent and detailed and impressive.

Cory Schires

Thanks. Again, credit to Danielle for marketing. At Scholastica, we all advocate for shipping features that are important to industry partners and improving our existing integrations to better support the flow of research. We like to stay connected with the broader publishing and research community not only to keep raising awareness of Scholastica, but also to support important initiatives like ROR, so it’s a win-win.

Amanda French

That’s great to hear. Even being fairly new to ROR, having been here a little over nine months, I could already tell coming in that awareness of ROR is not much of a problem, certainly not in the United States and Europe and Australia. We’re also doing a lot of outreach to Asia and South America, to the extent that we can, and that’s been terrific. But it has seemed to me that there is a lot of buzz about ROR. And I am glad to hear that it’s something you feel you can brag about.

Cory Schires

It solves a problem. And with all PIDs, the more people use them, the better they are.

Amanda French

That is so, so right. And while it’s always hard to say someone is absolutely the first, as far as we can tell, Scholastica is one of the first major service providers to do such a thorough integration of ROR, where it’s in more than one of your products. I’ve talked to a number of people at other companies that are thinking about it or working on it, and there are some other partial implementations or ones that are for specific customers. But anyway, congrats.

Cory Schires

That’s good to hear, though I am a little surprised. It was on our internal roadmap for, I don’t know, at least six months. It felt like we were saying “We’re going to get to this” for quite awhile.

Amanda French

The funny thing is that six months sounds fast to me! My certification has expired, but at one point I was a certified Agile product owner, and having worked on several software projects, especially in university libraries, I know that it’s very easy for things to sit on roadmaps for months and months and months and months. So, good for you! And now can you describe in general how you’ve implemented ROR in your systems?

Cory Schires

Sure. It wasn’t too difficult for us. One thing you already touched on is that we have separate products for peer review, production, and publishing. That means we had to put ROR support in at least two places: our manuscript submission form and the form that is used to create an article. So that was a consideration from the start. That actually kind of divided the project into two phases. First, you’ve got to collect the data; you’ve got to update your web forms, right? That’s the collecting part. Second, there’s using it – distributing it. Now that you have this data, you need to make sure that it’s getting into your Crossref metadata, that it’s getting into your JATS XML, that it’s getting into Portico and so forth. But anyway, you’ve got to make sure that anywhere you’re sending article data, ROR is going along for the ride.

And actually we did it in the reverse order. We did the second part first, which was to set up all our downstream integrations and different exports to understand and expect ROR identifiers, and we shipped that all the way to production. It wasn’t doing anything, but it was just there waiting to be useful. And then we did the first part, which was adding it to the various web forms so that we were actually collecting it. That allowed us to release it in two separate phases. You mentioned Agile project planning: we’re always looking for ways to break up an otherwise big project into a couple releases, if possible. So that’s the approach we took. That can be a good tip for anyone who’s similarly trying to keep this kind of project bite-sized.

Amanda French

Yes, that’s true: there’s the collecting of the ROR ID as one phase and the sending of the ROR ID as another phase. Actually, I know of several integrations that have done one or the other, not both. I know of people who are adding the ROR manually to their JATS XML, so they have people who as far as I know just copy and paste it into a template and send that to Crossref – smaller organizations. I also know of some systems that collect the ROR for internal use within systems and then don’t send it in DOIs to Crossref. The way you’ve done it is the ideal way.

Tell me, in your manuscript submission and peer review system, are you collecting ROR IDs for all authors or just for the corresponding author?

Cory Schires

The corresponding author fills out the form on behalf of all the co-authors, and when they’re filling that out, the fields for each author are the same – or mostly the same. There’s a little bit of difference for the corresponding author, but they’re mostly the same. And the ROR integration part is the same. So yes, we collect it for all authors.

Amanda French

And how do you handle when authors have multiple affiliations?

Cory Schires

Even apart from ROR you need your submission form to understand that authors may have zero or more affiliations. If your form is assuming each author will have one affiliation, that’s not terrible, but it doesn’t mirror what we know to be real in the world. So you really need that as a starting point. And I don’t think it’s particularly novel, but there’s a standard “Add another” link you can click. You’ll see that in different online tools. The approach we took feels fairly conventional in a good way, I think: you don’t need to get too clever with something like that. We accept the institution name, and that bit is the ROR name, and that field is what we use to find and associate the ROR ID. And then we also have the department field, which doesn’t really have anything to do with ROR, but that gives you a place to put the other information that you might want to put.

Amanda French

That’s exactly the way we recommend to do it, so that’s great. How do you handle it when an author is affiliated with an organization that is not in ROR?

Cory Schires

Going back to what I said before, you need to get those two fields. We call it “Institution” and “Department.” As people are typing into that “Institution” name field, we have a user interface element that people call a “typeahead.” I guess you know what that is.

Amanda French

I had actually never seen the term “typeahead” before I started at ROR, but that is what we call it in all of our documentation and our demo and our demo code. I thought of it as a “suggest box” or something.

Cory Schires

It’s an autosuggest sort of thing, you know, like when you’re doing a search, it suggests results. Anyway, so as you’re typing in the name field, we’re suggesting results in that typeahead. And if they click on one of those results, great: it will fill up the field, and we’ll have made the association under the hood. In the interface, you could choose to expose the ROR ID or put a green check or something, but we didn’t do that. We are just collecting it unbeknownst to the users. And to answer your question about organizations that are not in ROR, because it’s a typeahead field, they can also just type whatever they want. They don’t have to pick something from that list. We did think briefly about requiring ROR IDs, but we decided against that.

Amanda French

We recommend against that, because while we do have a process for people to request a new ROR ID for an organization, right now it’s just a Google form to request a single new organization. And it does take us roughly four to six weeks to get those new organizations into ROR, because we curate those requests. We have to check and see, “Is this a real organization? Is it in scope?” So we ask people not to require a ROR ID in forms, because otherwise we’ll get requests from people saying, you know, “I can’t submit my manuscript, and I have to do it by tomorrow.” So kudos to you: exactly the right way to do it. And we’d love to build a process for collecting some of this data from ROR users, getting information on what these organizations are that people aren’t finding in ROR, to see if we can add them. We’d like to do that someday, but it’ll take a while to figure out.

Cory Schires

We’d be happy to send you a CSV of organizations that people have entered that aren’t in ROR.

Amanda French

Feel free to do that – we’d love to look at it. You can email it to me. That’s where we are right now: we’re on the “email us a spreadsheet” end of the spectrum of data collection methods, which is fairly primitive, but we’d definitely be happy for you to do that. So, just a couple more questions. What benefits to you and your users is this ROR implementation providing?

Cory Schires

I would say the benefit to the users is getting better metadata. It’s an obvious answer, but users don’t always know about metadata. They also might not think of metadata as a feature, but we do. At Scholastica, we care about taking steps to enrich metadata – like adding ROR IDs, for example, on behalf of our customers, so they don’t have to worry about the technical aspects of metadata collection or creation and can instead focus on maximizing the discovery benefits.

Amanda French

Right.

Cory Schires

We really tried to integrate ROR into every possible way that we send data, to Portico or whatever the case may be; we really tried to add it everywhere. And if we missed a place, someone will tell us eventually, and we’ll add it there as well. But that’s the main thing, is just the richer metadata.

Also, and there aren’t any short-term plans to do this, but this does give us the ability to do things that people have asked for occasionally. For example, “Oh, I’d like to see a list of all my past authors, and I would like to be able to filter that by institution.” We couldn’t do that well without a stable identifier to know which institution is which, so I think it opens the door for some possible new features for us. It also might be interesting at some point to query the data to produce something like a pie chart showing the breakdown of the institutions authors are coming from by Carnegie classification, for example. I don’t know that that would be anything other than merely interesting.

Amanda French

Sure, yeah. It’s interesting that you’re not showing ROR IDs to the users, and that you’ve said that whether users care about it or not, the ROR ID and that enriched metadata is a service to them, and a feature, and it might enable other features. I think ORCID originally thought that they would be a behind-the-scenes piece of technical infrastructure, and I think we thought the same at ROR, that it would be mostly implemented under the hood in the way you’ve done it.

But there are systems that do display ROR IDs to their users, and it is kind of nice, because you can click on the link and go see some concise information about that organization. ROR is a bit different from ORCID in that ORCID records have to be maintained by individual researchers, so they do need to see it, whereas ROR IDs are not maintained by individual institutions. So I think you’re exactly right that users don’t really need to see the ROR ID, but they will benefit from that behind-the-scenes tidiness and enrichment of the metadata.

Cory Schires

As I recall, we did talk about that: “Should we show it?” We also talked about that case where the user types in an organization that is not in ROR; we thought about putting something like a yellow warning saying, “This isn’t in ROR.” We even actually started to code that, but then we pulled it out of the plan, because as we were thinking more about it, it was clear that people were going to see it and they would think something was wrong.

Amanda French

I have seen systems where they display a green checkmark or something similar when the organization is in ROR, and when they do that they add explanatory text and sometimes a link to the ROR website, so it can work, but we’ve had some misunderstandings from that kind of implementation.

Cory Schires

“These three authors have green checkmarks and this one doesn’t: what am I doing wrong?”

Amanda French

Yeah, it’s like, “Oh, no, nothing is wrong.” It’s a perfectly legitimate organization, or the author is an independent researcher, or, you know, there are all kinds of legitimate reasons for there not to be a ROR ID for an organization.

Cory Schires

And we can layer that in later if there’s some reason to, but this is where we’ve started.

Amanda French

What do you hope ROR does in the future? What opportunities should we pursue, or what would make it easier for you and other service providers to adopt ROR? What feature suggestions or bug fixes do you have for us?

Cory Schires

The main thing is that I’m just glad ROR exists. There are all kinds of reasons why you would want to know what institution an author is coming from. Before ROR, I was aware of some of the commercial competitors, but they were prohibitively expensive for many smaller journals and more difficult to integrate. We work with bigger journals and presses, but we also work with many small journals, and the price was a concern for our customers. When ROR came out as something that was free and community-supported, that was super important.

But that doesn’t really answer your question. I would say one thing I like about how y’all have set up ROR is your approach to parent-child or nested institutions. Obviously, you can go super deep on that and get extremely detailed. And I think y’all have helpfully resisted the urge to go deeper than makes sense, in my opinion. I mean, people don’t always realize that: it’s kind of counterintuitive that there can be an inverse relationship between the richness of a dataset and its utility. It can get so granular, and that’s cool, but then it’s really hard to make any kind of inference from the data. It becomes like Borges’s map. It’s really exact, but it’s useless.

When I was looking into that and thinking about that problem, I thought, “Oh, I like the approach they’ve taken here. I agree with this.” It made me feel like y’all are … well, the opinion aligns with mine, which is comforting to me.

Amanda French

Wonderful. I don’t know if you saw our recent blog post about hierarchies and relationships – we say some of those exact things. We found out that somehow people were under the impression that ROR didn’t do hierarchy at all.

Cory Schires

Yeah, that is what I thought initially. But I remember looking at the API and seeing, “Okay, yes, there are some relationships.”

Amanda French

You’ve been so kind about ROR when I asked you to give us constructive feedback! And hopefully, we’ll get to that. But honestly, I do think that is one piece of constructive feedback that we needed to hear, that we were mis-communicating about what ROR can do in terms of parent-child institutional relationships. People often ask us about university departments in particular, and we don’t do university departments, but that doesn’t mean we don’t do organizational hierarchy at all. We absolutely do that: for both parent-child relationships and related organizations.

That’s why we’ve just put out that explainer blog post, and we’ve also beefed up our documentation to say, “Yes, we do hierarchy and relationships,” so that you don’t have to look at the JSON in the API in order to figure it out. And we didn’t have the relationships in our search interface for awhile: we just added that a few months ago. Since a lot of people look at ROR just in that browser search interface, they didn’t realize the relationships were there. And we really do think that the degree of granularity we have, which is not as deep as some other organizational identifiers, is a feature, not a bug.

Cory Schires

I agree with that. Keep it simple, or at least relatively.

Amanda French

“Keep It Simple, Stupid,” right? The KISS acronym for that design principle.

Cory Schires

I might have more feedback on ROR as we start to use it more internally, but one thing I would say is that you should invest in the resiliency of the API and of the organization. This is a small example, and I love Crossref, don’t get me wrong, but we had some tests in our test suite that hit the Crossref API, and we had to disable those by default because sometimes the API goes down, and then our test suite is failing and causes problems for us. The ROR API is increasingly useful, I think, and we service providers need it to work for integrations.

Amanda French

That is great advice. Our API has had wonderful uptime so far, and we have a little API heartbeat, so you can just send a little quick query to that to see if the ROR API is OK. But we definitely have been seeing increased use, and increased use, and increased use, which is great. We want that to happen, but it does mean that there are more costs. As you might know, we are supported by three organizations, and DataCite is the one that pays for our technical infrastructure.

And I think organizational resiliency is part of technical resiliency. Last fall, we announced that ROR is now part of the operating budgets of Crossref, DataCite, and the California Digital Library. So we’re not going to have a member model, and as long as people are still paying member fees to DataCite and Crossref and tuition to UC colleges, then ROR will be around.

Cory Schires

One other thing about the API. As you’re obviously aware, you can get the ROR data dump and query that or you can hit the API, and we evaluated both approaches. Specifically, we were looking at, “Is this API going to be fast enough and reliable enough?” because we would prefer to hit the API than keep our own copy of the data, and in our testing, it was good. So that’s great.

Amanda French

And now that we’re updating the ROR registry about twice a month, if you’re using the data dump, you’ll have to make sure that you have the latest version of the data. You can get that programmatically by querying Zenodo. But if you’re querying the API, you don’t have to worry about that: you’re just getting the updated data automatically.

Cory Schires

Sure.

Amanda French

Is there anything else you’d like to say before we wrap up?

Cory Schires

One thing I would say is that this was a team effort. I didn’t work that much on the ROR project. Brian Cody, our CEO, did most of the project planning and actually most of the spec writing, which is not usually how he spends most of his time, but he wanted to help get this one done. And several different developers worked on it. Specifically, Tatum Szymczak really carried it across the finish line. A whole bunch of people worked on it, and that’s important.

Amanda French

Yes, it is! Well, thanks for talking with me, Cory. It was nice to chat.

Cory Schires

I appreciate your time. Thanks for involving me. Happy to be doing this for ROR.

Europe PMC provides comprehensive access to life sciences literature from trusted sources. It’s available to anyone, anywhere for free. Users of Europe PMC can search and read 42 million publications, preprints and other documents enriched with links to supporting data, reviews, protocols, and other relevant resources. Europe PMC partners with other organisations to build robust public tools to provide access to open content and data and to advance life sciences research.

Integrating ROR IDs into the Europe PMC Grant Finder means that organizational name variations are now aggregated under a single name, which allows people to search for different aliases and receive the same set of results. Europe PMC used the ROR API in conjunction with a manual screening step to match 82% of associated institutions in the Europe PMC grants database (GRIST) to over 2600 ROR IDs. ROR IDs are now returned as part of the GRIST API core response along with the institution name. The ROR ID is also used to auto-suggest a matching affiliation when a user searches for grants awarded to a particular institution by typing a name in the “Affiliation” field of the Grant Finder. This integration is now live at http://europepmc.org/grantfinder.

Europe PMC has supported ROR IDs for author affiliations since 2019, and in 2021 ROR IDs were added to a subset of publications by EMBL-affiliated authors as part of Project FREYA. Additional work for Project FREYA included the development of the ROR Predictor prototype, which enables matching author affiliations in research outputs with PMIDs to ROR IDs. Europe PMC also uses ROR IDs to enrich the DOI metadata of the research grants it registers with Crossref on behalf of the Wellcome Trust.

“Linking research outputs via persistent identifiers makes related information more findable and helps track provenance. Incorporating ROR IDs for grant awards supports DORA-compatible research reporting for funders and institutions who need to understand the impact of research,” said Melissa Harrison, Team Leader of Literature Services at EMBL-EBI.

Said ROR Project Lead Maria Gould of the California Digital Library, “Europe PMC has been remarkably forward-looking in its early use of ROR, and we are excited to see other services follow their example.”

See also the Crossref news release, “Europe PMC integrates ROR into its Grant Finder.”

Amanda French

We’ve asked Rob O’Donnell, Senior Director of Publishing at Rockefeller University Press, to tell us about some of the new things they’re doing with ROR.

Rob O’Donnell

Thanks for having me. Let me start by giving some pertinent information about Rockefeller University Press (RUP). We’re a small university press publishing 3 and 1/3 biomedical journals. The one-third is Life Science Alliance, which is a collaboration open access journal with Cold Spring Harbor Laboratory Press and EMBO Press. We’re predominantly institutional subscription based, transformative, and we’re using Read and Publish deals as a stepping stone on our way to becoming fully open access. We use eJournalPress (EJP) submission, production, and billing systems. Our content is hosted mostly on Silverchair; LSA’s content is on HighWire. We’re active with the OA Switchboard, which is a metadata exchange hub for both reporting and eligibility querying.

Rob O’Donnell

This slide is basically an outline of what I’m going to talk about, and I think it’s a good visual that shows how much our workflows now revolve around ROR. We’re using them really in every aspect of our workflow. I’ll start at the top with how we use ROR IDs to model our Read and Publish deals and then work clockwise.

Rob O’Donnell

We have Read and Publish deals with over 300 institutions across all our deals (totaling >600 ROR IDs). Now that includes consortia, but that’s the number of institutions we have so far. To get started, our Institutional Sales Manager, Miguel Peralta, mapped our subscriber IDs to ROR IDs. We already had Ringgold for most of our subscribers, so we started there and we went from Ringgold to ISNI to ROR. We also added those ROR IDs to our subscriber records in Silverchair’s SiteManager, and Emily will talk more about that later. Then we mapped archival author affiliations to ROR; we went back to 2017. Most of that mapping was done via Dimensions. We have a subscription to Dimensions, and Dimensions still includes GRID IDs in their results – not sure if they will update those to ROR IDs. They still use GRID internally, which made it easy to go from GRID to ROR. Once we had ROR IDs for both authors and subscribers, we were able to match our published output with subscribers in order to model and create our deal proposals. We also use the mapped ROR IDs to alert previous authors at deal institutions that they can now publish at no cost to them.

Rob O’Donnell

When we get new deals, we set them up in eJournal Press systems. They’re configured using ROR IDs, and the process is then completely driven by ROR. The system looks for a match between affiliation ROR IDs and deal ROR IDs. And once there’s a match, we know the article is eligible for that deal. Because we have consortia and large institutions, we need to map all sub-organizations that are included in deals as well, and this is a manual process. I’ve been wondering whether the IP Registry / PSI Registry could help here. They now have ROR IDs in the registry. We use PSI and they have our subscribers broken down and clustered into consortia or large institutions with only the sub-institutions as included in our subscriptions or deals. If those ROR IDs were included in the PSI API, it could be a huge help. When there is a match, the corresponding author is alerted that they’re eligible for a deal when they proceed to sign their license form. We are currently building other steps in the workflow to alert them earlier in the process.

Rob O’Donnell

When we first implemented this workflow we were only collecting ROR IDs for the corresponding author’s current address, which was a problem because that’s not necessarily a manuscript affiliation. Since then we’ve improved the process, and I show that in this short video. EJP has its own instance of the ROR database in their system. When the author is filling out their submission and starts typing the institution name the typeahead is looking up the ROR record in the EJP database. The author chooses the correct institution from the results list and is then presented with a green checkmark next to the institution name, an indication that it has been validated. We also have a new section asking the corresponding author for all of their affiliations. It’s the same process as just described for each affiliation. The video shows what happens if the author does not select from the typeahead menu, and they just hit Save, or if they choose a name that’s not in ROR – they get this message that basically says, “Look, if you leave it this way, you’re not going to be eligible for any free publishing.” Authors can add as many affiliations as needed, and all of those will be checked against our deals to see if the article is eligible. Our policy is that any corresponding author affiliation on the manuscript is eligible.

Rob O’Donnell

The next slide is about the OA Switchboard. If you’re not familiar with the Switchboard, I encourage you to look into it! Not only is the functionality important to publishers, institutions, funders, and (indirectly) authors, but the organizations involved are incredibly collaborative, experimental, and working with the goal of industry-wide solutions. I’m excited that we’re now live with our Open Access Switchboard eligibility workflow, which is known as “E1” in Switchboard lingo. If a manuscript is not recognized as part of a deal using what was shown in the last step, we use the ROR IDs we’ve gathered to sequentially query each of those organizations to ask if they’ll pay the gold OA fee for the article. We do this during revision when there’s a high chance of acceptance. We don’t do it for every manuscript. That would be overkill. We’re also querying contributing author affiliations, even though we’re not collecting ROR IDs for all authors yet. This is done using smart matching (originally developed by OA Switchboard’s tech team, ELITEX, and now available open source) and that’s working well, too. Smart matching maps text-based affiliations to ROR IDs (via the ROR API) and then directs the queries to those institutions. The Switchboard also maps our tagged Funder Registry IDs to ROR and queries those organizations as well. So, we’re tagging all affiliations, all funders, and asking them, “Hey, would you be willing to pay for this?” The actual question is “Does this intended publication meet your requirements/policies? If so, will you cover the publication charges?” and each organization can reply yes or no, and cover full or partial payment of the APC. This eligibility querying workflow has the potential to facilitate funding for articles that are not covered under a Read and Publish deal.

And of course, we also use the Switchboard, via ROR tagging, to report on published articles. The “P1” workflow allows participating institutions and funders to see where and when the research from their organization or research they have funded is published by an OA switchboard-participating publisher. Data from all publishers appear in a single dashboard, with the ability to export as consistently formatted Excel or JSON.

Rob O’Donnell

And just a quick mention: we’ve recently introduced a step in our production workflow where production editors will check for any missing ROR IDs. If they find that an institution is missing from the ROR database, they request that they be added to ROR. In some cases, authors like to use a medical school or a hospital if that is where they are rather than the parent organization. And that ends up coming across production without a ROR. So Adam, I’m sorry if you’ve been getting requests for med schools.

Adam Buttrick

That’s okay. That’s my job.

Rob O’Donnell

We have, in some cases, mapped medical schools to the parent ROR ID in the EJP ROR database so that, if someone starts typing a medical school, that name is in the typeahead, but the parent ROR ID is used under the hood. 

Emily is now going to talk more about what Silverchair is doing. And I’m grateful that they’re doing quite a bit! But briefly, we include ROR IDs in our JATS files, and after an article is published, Silverchair includes those IDs in Crossref deposits. As I said earlier, we also have our ROR IDs in Silverchair’s SiteManager, which checks entitlements. So the goal is to be able to report easily, via Silverchair, on Publish output and Read output for any institution.

And finally, we’re in a pilot with ResearchGate. Not all of our content is Open Access, and we are sending ResearchGate content that is access-controlled, so we need entitlement checks on the ResearchGate side. We do that via Google Scholar Subscriber Links and ROR IDs! With Anurag Acharya’s approval, Silverchair has added ROR IDs to our Subscriber Links and ResearchGate uses those to grant access to individuals from our subscribing and R&P institutions.

Amanda French

That’s amazing. That was really wonderful. I mean: all the applause. We will now hear from Emily Hazzard, Product Operations Analyst at Silverchair, about Silverchair’s part in this. Emily?

Emily Hazzard

Hi, everyone. I’ve just got the one slide, and we’re going to go into dark mode. Rob already talked about a lot of the things that we’re doing, so if there are questions, just let me know. But for those of you who I haven’t met, I’m Emily. I’m on the Product team at Silverchair. And my role is between the high-level visioning stage and the detailed requirements stage, so I’m involved in a lot of things ROR. We are mainly implementing things based on what our publishers find the most important, and we’re always open to ideas. I love persistent identifiers: I could talk about them for a long time. And that’s why we’re all here. So I’ll talk a little bit about what we currently support around ROR and a couple of things on the roadmap.

One of the things that we support is connecting institutions to ROR IDs. Publishers like Rockefeller are able to go into our SiteManager system, and then add ROR IDs to the institutional information that we store. This allows us to include ROR IDs in Google Scholar Subscriber Links, which Rob has already mentioned, and we’re going to be able to use those in other ways. Step 1 was making sure that we had the space for them in our system so that we can do everything that we want to do in the future. As you saw from that screenshot, we do currently include them in Subscriber Links, and that’s mainly being used to support the ResearchGate use case that Rob mentioned. This also means they are available to Google Scholar, but I don’t have a lot of insight as to what Google Scholar is doing with those ROR IDs now. If they choose to use them, I’m happy to share updates about that in future calls.

We also allow publishers to include ROR IDs alongside author affiliations in articles, proceedings, books, and chapters for the content that’s loaded to our publishing platform. That means that we’re able to then auto-magically downstream that information in Crossref deposits and PubMed deposits, as well as full-text deposits. Once we’ve got that information in our system, all of that information is going to be propagated out to all of those different indexing services, which is exciting. As far as things on our roadmap, in the institutional management service in SiteManager, we’re going to be introducing a way to search institutions based on the ROR ID just because it’s incredibly convenient. Sometimes it’s easier to find something based on a persistent identifier than something that might take a little bit of disambiguating (to find out how it might have been spelled or transliterated). Something else that we’re planning on doing is introducing ROR-based reporting to support those Read and Publish agreements so that we can expose the relationships between those authors and institutions, and really see what’s coming out of these institutions. Where is it published? Where does it live? And how do we serve those authors better?

So that’s what we’re doing right now. I’ve got lots of ideas for things that I am interested in. Again, talk to me if you have more questions. Are there any questions about the things that we’re currently doing or that’s on our roadmap here?

Amanda French

I suppose I’m a little curious as to whether all of the users of Silverchair get all of these features at once, or are you doing it more client by client?

Emily Hazzard

That’s a great question. So the SiteManager piece, adding ROR IDs to the institutional information, that’s available to any client right now who’s using that service. And then connecting authors to ROR IDs and including that downstream, that’s available to everyone on our latest version of the platform. We try to auto-deploy as much as we can, especially when we know it’s going to be abundantly useful for things like a persistent identifier, so that’s one of the things that I always try to make sure we can do where we can do it.

Amanda French

Wonderful. Thank you.

Emily Hazzard

Absolutely. Well, thanks, everybody. If you have any other questions, reach out.

Amanda French

Thanks for telling us all about your work with ROR, Rob and Emily.

Relationships in ROR at a glance

The ROR registry currently includes records for nearly 105,000 organizations (to be precise, 104,594 at the time of this writing). Across the entire ROR dataset, there are more than 22,000 total relationships between organizations (22,057), which means that approximately 21% of ROR records have at least one relationship to another record in the registry.

Parent-child hierarchies are the most common relationship type, with more than 17,000 instances. The most common scenario is that an organization has only one parent, but in some cases an organization has multiple parents (1,891 organizations, or about 1.8% of total records, have multiple parents). Several national-level and large-scale research organizations, such as the Czech Academy of Sciences (https://ror.org/053avzc18), have more than 100 of their child organizations enumerated in ROR. 

Since its first independent release in March 2022 as part of the planned divergence from GRID, ROR has created over 1,000 new relationships between research organizations in the registry. We do this in collaboration with the global research community through our transparent curation process, and this service is entirely free of charge.

Relationships in the ROR data structure

ROR records store both structural and temporal connections in the relationships field. Connections and relationships between records can therefore be mapped and graphed. The relationship types supported are Parent, Child, Related, Successor, and Predecessor. An organization can have multiple relationships, but each relationship must be classified as only one of the above types. 

The United States Department of Energy, for instance (https://ror.org/01bj3aw27), has many “child” organizations such as national research laboratories. Those child organizations themselves often have their own children representing different research units and funding bodies. Pictured below is a vertical “family tree” of the Department of Energy created from ROR records with an organization tree script showing DOE’s children and grandchildren (laterally related organizations are not shown in this view). This structure aligns with how the Department of Energy tracks both its research outputs and the research projects it has supported using ROR IDs and Crossref Funder Registry IDs. 

Relationships are displayed on individual records in the ROR search UI, as for example the record for the US National Cancer Institute, whose parent is the National Institutes of Health (NIH) and which has multiple children and multiple related organizations.

The relationships element in the ROR JSON schema is an array, so organizations can have multiple children, multiple parents, and multiple lateral relationships.

If an organization ceases operations and passes on its work to another organization, that relationship is also reflected in ROR through the “Predecessor” or “Successor” relationship type. For instance, the Canadian Polar Commission has been succeeded by the Polar Knowledge Canada, and this is reflected in the relationship metadata.

Curation policies and practices with relationships and hierarchies

ROR’s essential purpose is to connect research outputs to research organizations, and therefore it is not focused on capturing all subdivisions of a given organization, such as a university’s academic departments. The more granular the registry, the less suited it is to address the use case of tracking outputs at the institutional level. Likewise, large-scale reconciliation of research outputs, such as that done by OpenAlex to assign ROR IDs to millions upon millions of affiliation strings in their global academic graph, can become more difficult and time-consuming relative to the degree of depth. One of the things that users appreciate about ROR is that it provides a manageable level of granularity, which makes the dataset more immediately usable as well as easier to maintain than other sources of organizational data.

That being said, it is still important for the registry to be able to capture relationships and hierarchies between and within organizations. Through our curation process, we make careful determinations in consultation with the global research community about how to include and express these connections in ROR metadata. For example, when deciding whether a prospective child organization should be added to ROR or not, we consider factors such as whether it has an independent status/identity and funding sources from its parent organization, how the organization appears in affiliation usage, and whether a separate ROR ID for the organization would make the task of tracking outputs easier or more challenging. 

As a global registry made freely and openly available for users and integrators around the world and maintained through a transparent community-based process, we also consider regional specificity in curation decisions involving relationships and hierarchies. This includes being aware of linguistic variations – for example, government agencies in many countries (Australia, Canada, and others) are called “departments,” which is not the same thing as a university “department” in some countries – as well as the specificity of local institutional structures, such as the unique nature of research institutes in France’s higher education system. 

Another important aspect of ROR’s curation practices when it comes to relationships and hierarchies is optimizing ROR’s interoperability with other organization identifiers. ROR IDs map to other open identifiers such as ISNI, Wikidata, and Crossref Funder IDs. As part of ongoing work to reconcile as many ROR IDs as possible to these identifiers, we make decisions about adding certain child and related organizations that can help to optimize these mappings. This is particularly important in the case of the Crossref Funder Registry, which is expected to be incorporated into ROR in the future. Although we are proud that 90% of existing funding assertions in Crossref have been made with a Funder ID that already maps to a corresponding ROR record, we also understand the need to support the long tail of funding references and levels of hierarchy that are available in the Funder Registry, and we will continue to address this in the coming months. 

Use cases and current integrations

ROR users can and do take advantage of relationships and hierarchies in ROR for a number of purposes, including but not limited to the following: 

• Large, complex institutions such as university systems and multinational companies and nonprofits want to track research outputs for the entire system as well as for each constituent organization.

• Funders want to find all research outputs funded by the top-level agency or foundation as well as by subsidiary agencies and related research facilities.

• National funders and Open Access programs want to monitor both national and institution-level adherence to Open Access policies. 

• Publishers want to manage institution-level subscriptions and Open Access agreements, each of which may include or exclude particular institutional subdivisions.

One example of a system that has cleverly integrated ROR to enable finding research outputs both from a single organization and from related organizations is The Lens, which offers large-scale search of patents and scholarly works. In December of 2022, The Lens announced that it had added a new ROR-based feature: a field in its structured search called “Institution ROR ID Lineage” that lets a user find all the scholarly works associated with an organization’s ROR ID plus all the scholarly works associated with that organization’s children. The Lens was already using ROR in its institutional data model, but this fascinating new feature shows an appreciation of what ROR’s relationships field can do. Searching by the ROR ID for the University of London (https://ror.org/04cw6st05) alone produces just over 98,000 results, whereas searching by the ROR ID lineage for the University of London produces over 780,000 results from the University of London and all of its 21 children.

Germany’s Open Access Monitor, similarly, uses ROR’s hierarchies to allow its users to track open access data for large, complex German research organizations at the parent level, the child level, or in clusters of selected related organizations. The Max Planck Society, for instance, has nearly 100 subsidiary research organizations, and because the OA Monitor maps all the data it acquires from Dimensions, Scopus, and Web of Science to ROR IDs, it can give its users information about the open access compliance of all Max Planck Society organizations together, only one of them, or any combination of them. 

Finally, we are hearing from a number of publishers who are interested in using ROR to manage open access publishing agreements with research organizations. Our favorite example of a publisher who is already using ROR for this purpose is Rockefeller University Press (RUP). In a recent OASPA/JISC webinar on publishing workflows, representatives of RUP explained and demonstrated several different ways in which they use ROR, including for the purpose of managing open access agreements. Their systems compare the ROR ID of the corresponding author’s affiliation to the ROR IDs for agreements, and when there is a match, the author is notified that their work falls under the terms of that agreement. RUP’s implementation shows the value of having ROR IDs incorporated in multiple parts of a publishing workflow and demonstrates that ROR’s degree of granularity is already more than sufficient to increase the efficiency of such workflows. Watch the full RUP presentation and/or the full webinar below:

We hope the preceding examples can serve as models for other projects that might use ROR’s relationships and hierarchies for similar purposes. If you too are using ROR’s relationships or hierarchies, we’d love to hear more about it – tell us more about your integration and we’ll feature you on our website, or get in touch via support@ror.org if you need help getting started.

Final takeaways

ROR’s support for relationships and hierarchies is specifically informed by our core use case of connecting research organizations to research outputs and motivated by our aim to provide an open and easy-to-use dataset with inclusive global coverage. 

We hope this post has helped to provide more detail about how relationships and hierarchies are supported in ROR, how we make decisions about relationship structures in the curation process, and how this metadata is being leveraged by users and integrators. 

To learn more, check out ROR’s technical documentation, and get in touch via support@ror.org with any questions. If you have feedback about relationship structures to add to or update in the registry, let us know via the curation request form.

This post was jointly written by members of the ROR core team: Adam Buttrick (Metadata Curation Lead), Amanda French (Technical Community Manager), Maria Gould (Project Lead), and Liz Krznarich (Technical Lead).

Amanda French

Thank you so much for joining me to talk about your ROR implementation. If you could, please start by telling us your name, title, and organization.

Tom Morrell

Hi, I’m Tom Morrell. I’m the Research Data Specialist at Caltech Library.

Amanda French

Perfect. Can you tell us about Caltech Library and the unit that you’re in?

Tom Morrell

Caltech Library serves all the information needs of Caltech as a campus. And Caltech is a kind of unique institution. We’re small in terms of numbers. We have about 300 faculty, but we have a really big impact in a lot of research areas – in Astronomy, Astrophysics, Biology, Neuroscience. We’ve got a lot of high-impact work, just done with a very small staff. So what the library does is we serve everybody on campus. We serve traditional library information needs in terms of journals, for instance, but we also have a very strong repository program in terms of making the research content of Caltech publicly available. That includes publications (CaltechAUTHORS) and theses (CaltechTHESIS) as well as what I spend most of my time working on, which is datasets and research software (CaltechDATA).

Amanda French

And when did when did Caltech first start its repositories? You have both an institutional repository and a data repository, correct?

Tom Morrell

I don’t remember off the top my head the original date. We were very early. We were part of some of the initial discussions up at Los Alamos about the concept of institutional repositories and making content open. We have over 100,000 items in our main institutional repository, which, as a percentage of the output of the institution is very, very high, so we’ve got lots of dedicated librarians who’ve put a lot of effort into collecting content and writing up quality metadata for the content and making that publicly available. And one of the things that I really enjoy being able to do is to figure out ways of making that content more discoverable and more valuable, using things like ROR.

Amanda French

Wonderful. So where and when did you personally first hear about ROR?

Tom Morrell

Oh, that’s a good question. It was probably at one of the conferences very early on, when ROR was just beginning, and it was likely in the context of DataCite metadata. That’s where I’ve spent a lot of my time, is figuring out how we can best describe things like our datasets and software in DataCite metadata. And one of the challenges always was, you know, How do we do affiliations? And before ROR there wasn’t really an easy, open way of saying “This the affiliation that I mean.” So when I heard of ROR, I was like, “This is really just the perfect solution for how we handle affiliations and identifying where people are coming from.”

Amanda French

Were you the primary advocate of implementing ROR at your organization?

Tom Morrell

I would say yes, but we had lots of other librarians who were also very excited and interested in this. Because it’s going to be helpful not only for discoverability by making our metadata better, but it’ll also be really helpful for reporting. One of the types of reports that we have to do often is compliance reports for funders, and we track who’s collaborating with who. At the moment those are challenging to put together, because you need to know everybody’s affiliation, and trying to match up what a given affiliation is when you have random strings that are coming in from publications is really challenging. So we have librarians who are very excited about the possibility of actually having identifiers for these affiliations. That makes a lot of the reporting stuff a lot easier.

Amanda French

And just to clarify your role: are you responsible primarily for the data repository, or do you also have a role on the institutional repository?

Tom Morrell

My primary role is the data repository, but I also help out with the institutional repository as well on the technical end. We’re currently in the process of migrating the institutional repository (CaltechAUTHORS) to InvenioRDM, so I’m leading the group that’s doing that migration.

Amanda French

That’s actually very exciting.

Tom Morrell

So yeah, I kind of am the bridge. We have research services librarians who work with the researchers and make all the decisions for the institutional repository: How should we describe these articles? How should we tag and organize and collect them? And they enter them into the system. Where I come in is more on the technical end: How should we actually manage the metadata? How can we do bulk updates to the institutional repository? And then we’ve got software developers in the Digital Library Development group that do more of the you know, nuts and bolts of really doing the software development. So I’m kind of in an intermediate role between the librarians and the software developers.

Amanda French

Gotcha. So your thesis repository, I believe, currently runs on EPrints, and your data repository runs on InvenioRDM, and you have ROR integrated into both right now?

Tom Morrell

Yes, we do. I’ll run through the history of how we did the integration. Basically as soon as ROR came out and it was supported in DataCite, once they said, “We’ve got a spot in the metadata to put ROR,” we pretty quickly implemented ROR for our thesis repository. That was a really easy, easy case, because we know everybody who has a thesis in the thesis repository is affiliated with Caltech. [Note: For an interesting analysis of affiliations in university repositories, see Habermann, T. (2022, June 24). Universities@DataCite. Metadata Game Changers blog.]

Amanda French

It’s one single ROR ID!

Tom Morrell

It’s one single ROR ID. And for the DOI minting, we were not particularly happy with the built-in plugin. EPrints generally, we’ve used it for a very long time, and it’s been a great system, but it is definitely showing its age. So we started doing some of our development work outside of EPrints, because that allows us to be much faster and more efficient than writing things in Perl. So the way we do our DOI minting for the theses, we basically have an external script that pulls out the metadata, reformats it, and then mints the DOI. So we were able to pretty easily add ROR IDs for the theses. Each thesis has an author name, and then we put in the affiliation with the ROR ID. And then we went through and then bulk updated all of the DOIs with the new affiliation and ROR ID. That was our first implementation, and it was very straightforward because we knew what the ROR should be, we knew where it would go, and we had the tooling to basically automatically add it.

Amanda French

About how many theses do you think that was, that you added the ROR to in the DOI? Do you know, roughly?

Tom Morrell

It was approximately 10,000 theses.

Amanda French

Fantastic. I’m gonna ask you a very random question now, and there’s no shame if you answer it in the negative. Do you by any chance have the ROR ID for Caltech memorized?

Tom Morrell

I have about the first 3 characters. I don’t have the full one memorized.

Amanda French

It’s meant to be random, so I’m not surprised, but I just wondered, since you’ve had so much experience with that particular ROR ID. So that’s ROR in CaltechTHESIS. Tell us about the data repository and InvenioRDM. How did you decide on that system? And tell us about ROR in that system. [Note: CaltechDATA on InvenioRDM launched in September 2022.]

Tom Morrell

When we were looking at the data repository we wanted a system that was open source, but also one where it was really easy for researchers to do their own deposits. In comparison to the institutional repository where you get the publication, and you can look at it and say “This the title, and these are the authors” and so forth, we have a random data file. You really need the researcher to be involved in the process of describing what the dataset is and what papers it’s connected to. So we wanted to make sure the researcher had a very easy and straightforward way of making a submission to the repository, and one of the things that we liked about Invenio and InvenioRDM is it’s based on the really successful open repository Zenodo, which has gotten a lot of traction, because it’s very easy to go upload your dataset and get them DOIs and get them preserved in a way that other people can access them. So we really like the ease of use, and we like that it’s an open source project, and we like that it was written in Python, so it’s easy for us to do additional development and make additional contributions to the community. And one of the exciting things for us about InvenioRDM has been to see how the community has grown over the last couple of years. We were one of the first members in the group, and it’s now got lots of new people joining and working on rolling out their own repositories for their own institutions. So from a technical side, it’s nice to be able to make contributions into a codebase that then a lot of other people are going to use.

Amanda French

Yeah, I can imagine that. And Zenodo just announced, I think a month or two ago, that they are officially migrating to InvenioRDM themselves.

Tom Morrell

It’s very exciting that that’s public now and that we’ve got a more firm timeline for it. There’s a lot of nice harmonization if folks are working together on a project. It makes everything more efficient.

Amanda French

So my understanding is that InvenioRDM has ROR basically already built into every part of it. Did you have to do any additional customization to it?

Tom Morrell

Yes. I’ll talk about our migration process. There’s what we did with our data, and then there’s what we hope to be able to do in the future with InvenioRDM. We had an initial version of the data repository which was on another version of Invenio, and in that one we didn’t have any affiliation identifiers. There was an affiliation field, but no identifier, because we implemented this before ROR existed, and we didn’t like any of the other identifiers that were available. So as part of the migration, we wanted to start to add RORs into the existing affiliations that we had. And librarian Kathy Johnson did a great job. We used the RORRetriever from Metadata Game Changers.

Amanda French

Ah, yes.

Tom Morrell

We used that little automation to take all of the strings that we had for affiliations and make a guess as to what the ROR might be. And then Kathy went through and verified whether that was correct or not. So we did kind of a minimal set of data enhancements, finding RORs for our existing records.

And then in the configuration of InvenioRDM, you can basically load in ROR for both the affiliation field, so all the affiliations, as well as for funders. So we’ve done that. When you go in to add an affiliation, it does a little search, and it’ll auto-complete based on ROR. And then similarly for the funders it will auto-complete the funder name and you can click which one you want. So that’s just out of the box configuration.

In terms of integration, we did some data cleanup, we configured InvenioRDM how it works out of the box, and that all is great. But we also have a lot of improvements that we want to do for the future that we hope we can contribute back, and things that the InvenioRDM community in general wants to work on. There’s being able to pull in the latest changes with inactive versus active RORs and have that show up in the interface, there’s having the country show up so we can get better disambiguation for organizations that have the same name, stuff like that. So there’s still a number of areas where I think we can improve InvenioRDM and how it handles RORs. But it’s been built so that the bones assume that ROR exists, and that’s really nice, because we’ve got that open identifier baked into the software stack.

Amanda French

Sounds great. It really is a terrific community, too, InvenioRDM. I lurk on the Discord, but I have not been hugely active in it.

Tom Morrell

The community is the best part of it.

Amanda French

That’s great. So can you tell me some of the benefits of having ROR in both the thesis repository system and the data repository system?

Tom Morrell

Yeah. In terms of the thesis repository, it’s allowed Caltech to be much more visible. If you do a search in DataCite for Caltech, you will get all of our theses, and it’ll pull up the disciplines and the tags. You just put it in the system, and it just works, because we’ve attached the ROR to all the affiliations. So in terms of accurately describing the output of the institution, it’s helped hugely in terms of just accurately saying, “This scholarly content has come from Caltech.”

In terms of the data repository, we’re still figuring out where it’s going to be useful in terms of reporting and cleanliness. One of the areas where it’s going to be very helpful is in terms of funders. At the moment we don’t have to do a ton of reporting back to the funders to say what datasets were associated with a given grant, but we’re anticipating that we’re going to have to do that in the next few years, so having the ability to pull out exactly what records are associated with a given funder is going to be very important. We have it in the system now, so instead of just having people type in the text and the different variations of “National Institutes of Health” or “National Science Foundation,” instead, you have them type it in, it assigns the ROR to it, and then we’ll be able to pull records out via the ROR. We don’t have as many specific use cases where we can say, “Yes, this is the reporting we’re using this for today,” but we’re anticipating it’s going to be really, really helpful, because there probably will be more compliance reporting we have to do.

Amanda French

Great. So what problems have you identified with ROR, if any? What do you wish ROR did that it isn’t currently doing?

Tom Morrell

One of the challenges we’ve found is in terms of what name shows up as the preferred name. Currently there is a little bit of variability as to what name is the main name for an organization versus other names. Particularly for international organizations, you know: Is the English name the one that shows up in the preferred name, or is it the Spanish name that shows up in the preferred name? We’ve run into some of that when we were doing some of the cleanup work, figuring out which ROR is really the right one for a given affiliation. And that’s something that also comes up in InvenioRDM when we’re going back and looking at old records where we have just affiliation strings. We have to make a decision. Do we go with the name that is associated with the ROR or do we leave the name that somebody typed in? There’s a decision point.

Amanda French

Right.

Tom Morrell

How do we respect the data that the user entered in, which might not be the cleanest or in the right form that we want, but also not change the way the name is shown? That’s something that we’re still working through, how we make decisions about how the name shows up in the system and on the specific records.

Amanda French

Oh, that’s interesting. Most ROR records have a lot of different names in different fields. There is the primary name, which, because we inherited it from GRID, tends to be in English. I’m not sure if they had that as a policy or if that was just their practice. But we definitely don’t have that policy, so we’ve been taking lots and lots of requests from international organizations to have the primary name in the name field be in the primary language of the country where that institution is located, when that is not English. But yeah, that’s interesting. And I’m sure you’ve seen our schema change proposals. We have lots and lots of proposed name-related changes.

Tom Morrell

Yeah, you’re all working on it, so it’s not like I’m worried about this at all. I think you’ve got it all under control. But that is an area where we did experience some challenges in terms of the data. And we have some general challenges with InvenioRDM in figuring out how we do the display of things.

Amanda French

Yeah, and even then, I’m not sure what we could do to help with that display issue, really.

Tom Morrell

No. It’s on our end to figure out how we want to handle that.

Amanda French

Interesting. I think that’s about all I have for you. What else do you want to tell us about the data repository, or the thesis repository, or your integrations of ROR generally?

Tom Morrell

I don’t know if I have anything else. I’m a big supporter of ROR, and I’ve been really thrilled to see how it’s evolved, particularly over the last couple of years. I feel like ROR has really gotten into its stride in terms of how it’s managing the curation process, making the data available, providing both useful data dumps and API features. I think it’s going to be a really powerful tool for allowing us to do all sorts of queries and identifying how people are associated with organizations and funders.

Amanda French

Great! That’s great to hear. Thank you so much, Tom.

Tom Morrell

Well, thank you. It’s been great to chat.

Since then, we have continued to celebrate ROR’s anniversary every year with a big public event to bring together the broad network of ROR users and supporters and reinforce ROR’s commitment to developing open, sustainable, community-driven infrastructure. (In previous blog posts, we’ve shared recaps of the 2020, 2021, and 2022 meetings.)

For the 2023 annual meeting, we celebrated four years of ROR with four exciting virtual sessions attended by hundreds of participants from around the world.

Session 1: ROR Community Update

In the first sesssion - a general community update - the ROR team reviewed key highlights from the past year and discussed plans for the coming year. We also shared ways for community members to get more involved in ROR activities (if they are not already participating).

Session 2: Open Drop-In Hour

In the second session, we held an open drop-in session (not recorded) to meet informally with anyone interested in chatting about ROR. This is a model for similar sessions we’d like to hold throughout the year, especially to be able to reach people located in different timezones or who are not always able to attend the bimonthly ROR community calls. Stay tuned for announcements about future drop-in times!

Session 3: Introduction to the ROR API

This year’s meeting also featured the introduction of a new session focused on how to work with the ROR API. Technical Community Manager Amanda French delivered an engaging tutorial on how to use the ROR API and get up and running with basic queries. If you weren’t able to attend the session, watch the recording and check out the workshop materials!

Session 4: Strategies for Matching Affiliation Strings to ROR IDs

The last session, “Strategies for Matching Affiliation Strings to ROR IDs,” featured a panel of users (OpenAlex, Allen Institute for AI, DOE/OSTI, and Crossref) sharing their work to develop automated approaches to reconciling affiliation data with ROR. We are eager to bring people together to develop collaborative solutions to shared problems, and the panel was a great example of putting this into practice.

Four more years - and then some more!

Four years is a huge milestone for ROR! At this stage in our journey, we are proud and excited that ROR’s open data and API are being used around the world and in more than 50 known integrations, that registry records are being actively and regularly maintained through a transparent community-based process, that ROR is supported in Crossref, DataCite, and ORCID and being recommended in national PID strategies (most recently, in Australia), that ROR’s three governing organizations are committed to ROR’s long-term sustainability, that ROR has been selected by SCOSS, and that we are part of a growing movement of initiatives to develop and follow best practices for community-led open infrastructure.

We could not have reached this milestone without your support - whether you have been part of ROR since the very beginning, or whether you are a newer member of the pride. Thank you! Here’s to ROR-ing together for the next four years and then some more!

Amanda French

So, Allyson, please tell me your name, title, and organization.

Allyson Lister

I can do that. I am Allyson Lister, and I am the FAIRsharing Content and Community Lead. I work with Ramon Granell, who is a Research Knowledge Software Engineer, who unfortunately isn’t able to join us. We’re both staff at the University of Oxford at an institute called the Oxford e-Research Centre (OeRC), which is a cross-disciplinary center.

Amanda French

So can you tell me both about that Institute and about FAIRsharing? What do they do, and what makes them special?

Allyson Lister

The OeRC started in 2006, and was created as a cross-disciplinary research Centre in digital methodology, working across the humanities, communications, engineering, life sciences and more. And for FAIRsharing, we started in about 2009 as the result of an article in Science that was authored by a number of different research councils, research institutes, and funding agencies and led by my boss, Professor Susanna-Assunta Sansone. As a result, BioSharing was created, which was intended to collect all the existing data policies within the life sciences to ensure they could be compared and harmonized according to the resources they recommend. You know, the standards they recommend, the databases they recommend. And this was so popular that it changed in 2011 from this original blog site to a website launched from the University of Oxford. And so it continued to grow over the next few years, including under the auspices of an RDA working group. That’s the Research Data Alliance. Are you familiar with them?

Amanda French

Yep.

Allyson Lister

They are a fantastic, absolutely amazing, volunteer-led community organization for research data. And so that went along quite merrily for a few years, until the publication of the FAIR principles in 2016, which Susanna and two other members of our group at the time were authors on, among a bunch of other authors. And so, when the FAIR principles were published, we quickly realized that the best thing to do was to change the scope of BioSharing to be more than life sciences. And so it became FAIRsharing in 2017. And now it’s cross-discipline, looking to store information about databases, standards, and data policies across all research areas, because these are really the pillars of the FAIR principles; they give you the ability to describe the places you store your data, the way in which you format your data, and who is asking you to store and structure it in a certain way (such as funders, journals, societies). And so this is where we are now: a curated registry. Or, in reality, a curated set of three registries for (meta)data standards, databases, and data policies.

FAIRsharing doesn’t just describe those resources; my favorite part is that it links them together. So it creates this network of all the relationships, the landscape of resources for a particular domain. It allows you to explore what’s available for your particular research area. And it’s been adopted not only by the RDA, in which it continues to be an active member, but also a number of other international organizations and funders, such as EOSC and FORCE11. We have all these different organizations and publishers – journal publishers, funders. And the operational team is advised by Executive and Stakeholder Boards that we use to make sure our direction is right for the community that we serve. And I think that’s about it. Later on I’m going to touch upon my favorite bit, which is that I’m running this community curation program that is bringing our domain experts even closer into the project to help them showcase their resources.

Amanda French

Oh, wonderful! So where and when did you personally first hear about ROR?

Allyson Lister

I felt like I’ve always known about ROR, at least ever since we started looking outward. The first few years, it was all about curating the resources. So we were very focused on the databases and standards and policies, and we always linked to organizations. But we didn’t have any way of unambiguously referring to them other than our own internal curation checks. And then at the beginning of 2022, when we redeveloped FAIRsharing and created a brand new data model and a brand new front end, we said, “What we need is for organizations to be first-class citizens.” And so I immediately thought of ROR. Because what you get is an unambiguously, persistently-identified set of research organizations that will fix a lot of the problems that we had found. It wasn’t a huge issue, but there were cases where we had redundancy in our organizations. And so we did a big push to align our organizations with ROR organizations.

Amanda French

Right.

Allyson Lister

FAIRsharing is a part of the Data Readiness Group. As well as working with data transparency, we also look at research integrity and making data FAIR for humans and machines. And so at FAIRsharing we are looking to incorporate many different international community activities around research data management. So ROR was a natural place to go to for organizations.

Amanda French

Well, it’s interesting, because I think the timeline of FAIRsharing and ROR overlap quite a bit, because the planning for ROR began really in 2016, and went on through 2019 when it officially launched, which is sort of right when FAIRsharing was making the move to become FAIRsharing.

Allyson Lister

Exactly. We became FAIRsharing officially in 2017, but by 2018, 2019 we were really starting to snowball into the other communities. The life sciences had already heard of us, because we are a part of ELIXIR. We are a recommended interoperability resource in ELIXIR, which is one of the parts of EOSC Life within the broader EOSC context. I was at the EOSC symposium in Prague a few weeks ago, and everyone was clear about the interoperability aspects being really important. And I think that’s something that you need to be very proud of at ROR, because you are the one connecting and making sure that people are saying what they mean to say when they’re talking about organizations.

Amanda French

And that does lead into a question that came up when you were talking about your own awareness of ROR which is: did you consider any other organizational identifiers when you were refactoring FAIRsharing?

Allyson Lister

You know, I’m trying to remember if we did, or if once I heard of ROR, it just made sense. Ramon has told me that your API was really easy to use, and this is really important when developing an integration like this one.

Amanda French

That’s very good to hear!

Allyson Lister

He said the API was never the issue. The issue was the classical natural language processing issue, you know, of comparing, making the initial comparison against what we already had from 2011 onwards to what you guys had. But I’m trying to remember if there were any other major contenders. I have to say that ROR is the one that you always hear of in presentations. ROR is the one that you’re always referring to when people are talking about it, and it just seems … I don’t know exactly what your perspective is, but from my perspective it’s the de facto way of describing organizations. And there are bits of the ROR scope that don’t align with FAIRsharing. I’m not saying that’s a criticism of ROR. There are some places where we can’t match our organizations to ROR, and that’s not a limitation as much as just an acknowledgment of the differences in the way that people describe an organization. Because you could go on in workshops for days about “What is an organization?” And I’m sure you have, being ROR! So it’s just about deciding what you want to describe, and clearly listing what those requirements are, and for ROR that’s very clear. You go the on the website and you can click straight away and find out what your scope is.

Amanda French

Wonderful. So let’s get into the “How” at this point. Describe how you’ve implemented ROR in FAIRsharing. Tell us about that.

Allyson Lister

Well, in our old version we had a very nice web form that a lot of people used, that allowed you to “free-text add” everything about an organization, and while that organization could be reused, it was an object local only to FAIRsharing. Equally, while there weren’t a hundred different versions of “University of Oxford”, we did see duplications and mistakes creeping in. This was also true for ORCID IDs before our integration as a trusted partner with ORCID and for our subject areas and our technical domains, which were free text in the earliest versions of our project. So we wanted to align with community ontologies. We wanted to link up with ORCID. We wanted to link up with ROR. So this was our opportunity when we re-wrote the data model. You know, I wrote in my notes for this question that I wanted to mention xkcd and Randall Munroe. I have to!

Amanda French

I know the one you’re talking about, of course!

Allyson Lister

Yes, the reinventing of the wheel. The 14 standards we have, we must make another one to align them all, and then we have 15 standards. We try our hardest when we’re all aligning, wanting to align with FAIR principles, to ensure that we’re not making this kind of mistake. Inevitably there’s going to be some, but we do our best at every stage not to do that. So our subject tags and our technical domain tags all come from community ontologies. We use nearly 60 different community ontologies that we’ve merged together to try to create all of the tags that our users had already put into the system. So you know, most people would say, “Let’s find an ontology we like, and that’s what we’ll provide our users.” But we had had seven or eight years of users putting whatever they wanted, self-describing their records.

Amanda French

Right.

Allyson Lister

And so we were working from the bottom up, and so we ended up having to look in lots of different places. But for organizations, thankfully, it was more straightforward. We had a lot of organizations. Our definition of organization was quite deliberately loose. And so when we went to ROR, we couldn’t match everything, but it was still a useful match for what we did have. So not reinventing the wheel, using community terminologies, linking up with ORCID, linking up with ROR. FAIRsharing records can have maintainers. We also have our community curators, and both of these are the people who can edit records. So you either have to be the owner of the resource – a standard developer or a database developer or a policymaker – or you can be one of our domain experts that we’ve given particular roles and can edit multiple records. And they are trying to describe their resource in a way that is pertinent to them in their community. And so we try very hard not to put too many requirements on what they have. We provide fifty, at least, maybe sixty different metadata fields, but only something like three of them are required, because we understand that for each community, especially when we’re working cross-disciplinary, they’re going to have different needs for what they need to describe. But, saying that, we always like to have an organization. So even though it’s not a required field, we’ll come back to them and say “We’re not going to publish it till you put an organization in,” because we want to explore with them what they think of as an organization. So they can say that an organization maintains a resource, or collaborates on it, or funds it. A general “associated with,” if it’s sort of nebulous and they still want to include someone. Like often EOSC Life or EOSC will get an “associated with,” because your project is part of and supported by them, but it’s not a direct connection. And so we have a lot of different relationship types that link our records to our organizations. And so ROR being a large, open, community-driven database of organizations that has lots of different fields, and having relationships among the organizations, incorporating ROR into the new data model helps us speed up the creation of organizations in FAIRsharing for our users.

FAIRsharing doesn’t just allow them to search ROR while they’re making a record, but also provides the metadata from ROR for them so they don’t have to put it in themselves. So it’s a really nice way to ensure that we’re accurately describing what they need. Now in terms of direct implementation, what we did is – or I shouldn’t say, what “we” did. I helped, Ramon mainly did, and the rest of our curation team backed him up. So we have something like 3,500 organizations in FAIRsharing, which is nothing compared to, what is it? A hundred thousand [in ROR], or something like that?

Amanda French

About 104,000 right now.

Allyson Lister

A hundred and four thousand. Yes, and so clearly we’re not an organization registry, but people over the years have added many organizations. Ramon took our organizations and pulled the information from your API, and calculated an ’edit distance,’ so he looked at the name and the abbreviation and the home page fields for ROR and for FAIRsharing and he calculated the distance between the two terms for all of them, finding the least distance, the closest match. [*Note: More information is available in Ramon’s blog post about this work, and Ramon has made the scripts available in the ror-utilities GitHub repository.]

Amanda French

Right.

Allyson Lister

And then he provided a set of spreadsheets, where the majority have a cutoff distance. So if the edit distance was a certain value or or lower, then we didn’t really have to look at it and could discount them immediately. At the other end, there were a few hundred high-scoring matches that were very clearly matches. And then the curation team focused on those where the edit distance started moving away from optimal, but wasn’t so far away that it was clearly nonsense.

Amanda French

So this was a one-time process?

Allyson Lister

This was a one-time process, but now gets repeated on a smaller scale, I think it’s every three months. Because we always have new people adding organizations, and of course we encourage them to add organizations through the route we have with ROR. That means they should search ROR, because we have it built in, we have the connection to your API in our edit interface. But they don’t have to do that route when they put an organization in, and so every three months we run everything again, and we check all the new matches or any changed matches, and make sure they still make sense. So it’s still something that has to carry on, it will have to carry on for as long as we integrate with you, or we’ll just draw away over time.

Amanda French

That’s really fascinating, actually. And I love doing these interviews, because I’m always hearing about new ways to solve different problems. So first of all, that is one thing that we recommend, that nobody require the entry of a ROR into their system, because there certainly are institutions that are not in ROR that should be, or that are not in ROR because of scope issues, but that are in scope for your system. Oddly enough, you can require an organization without requiring a ROR ID, right, but you’re not even requiring organizations, correct?

Allyson Lister

We’re not, but we generally don’t allow it unless they put something in. So it’s a soft requirement, shall we say.

Amanda French

Yeah. But what is really interesting is what people do with those things that don’t match a ROR. I haven’t actually heard of anybody doing this kind of repetitive batch matching before. I know there may be other people doing it, but I think that’s really fascinating.

Allyson Lister

Thank you! Others seem to agree with you, as Ramon presented this work last fall at the Research Software Engineer Conference! If we’re going to integrate with somebody like ROR, we have to commit to it, you know. It was intensive at first, but mainly because of the creation of the original scripts and the batch of curation checks, as our records are all curated. But we’ve had a couple of these three-month checks since, and it’s been an hour or two maximum of checking, with the main curation team doing maybe fifteen minutes or so of checking each. So it’s not been a high overhead since then, and I think as long as you keep it up, you’re fine. But what we found is often that people can be overly verbose when they describe an organization, and they include, for instance, the name of the organization and all of the abbreviations in the same line and the same string, and maybe even the location, like a state and the country, or a city and the country. And they might use foreign language words. Some resources naturally have, as you know, working in ROR, multiple different names, depending on whether it’s in their native language, or in English, which they often add. And then home pages change over time, so sometimes the home page we have wasn’t the same as yours, even though the name was the same. Or sometimes the home page was the same, but one had an “https” and one had an “http,” or one had a “/home” and one had an “index.html.”

Amanda French

Yeah. Yep.

Allyson Lister

All of these different calculations. And so the automated procedure can only go so far because it will find a decent edit distance if there’s these ancillary changes, but ultimately the core is the same. So it’s interesting to really see how it works. And so, as of this month, Ramon’s given me some numbers: he says we have 3,462 organizations, and of those just over 1,500 are mapped to ROR, so we’re closing on 50%. But I don’t know if we’ll ever be going much higher than that. I think it’ll probably sit around that because of the scope issues. For us, 50% is not a “failure”.

Amanda French

I’m sort of interested in that: whether these are organizations that should be in ROR or whether it’s primarily a scope issue.

Allyson Lister

I don’t actually know. I’ve forgotten, do you include companies?

Amanda French

Yes. We do. The companies that we tend to include are companies that produce research, and that researchers will cite as their affiliated organization when they publish research. So because the primary use case of ROR is researcher affiliation – certainly not the only way that people use ROR at all, but the primary one – we include 3M and places like that, where people conduct and publish research affiliated with that company.

Allyson Lister

Yeah. So things like laboratories. A lot of people want to say that my group at this university is the one who made this, and so we allow those. Consortia. So with the International Virtual Observatory for astronomy. They make a lot of standards, and while not really an organization, they are the ones who are responsible for it. So we need to talk about them as an organization. And there are these programs that are part of the government, for example the Canada Vigilance Program which is related to the COVID-19 dashboard there. These programs are part of a government group or institute, but we allow them to self-identify. And so what we define as an organization becomes nebulous, or perhaps intentionally fuzzy. We said, “We’ll take all the ROR organizations that we want, but we’ll also have these other organizations outside of ROR’s scope, because this is how people want to provide attribution for resources.” And it’s a kind of personal decision. They feel, and I think rightly, tied to who owns a particular resource. And some will just put in their university, while others will put in the university and the department and the laboratory – all these different levels that they want to show. And so we try to be agnostic about what it is, and let them choose for themselves.

Amanda French

That makes total sense. And as you doubtless know, we inherited our seed data from GRID and have just begun curating the ROR registry independently in the spring of 2022, so we’ll see where it goes. Labs are in scope for ROR, but I suspect that we’re missing a lot of labs that would be in scope for ROR. And then for a lot of labs in a university, I mean, you know, there are sort of official ones, and then there are just sort of research groups that call themselves a lab that may not have an official existence.

Allyson Lister

Yeah. See, that’s interesting, because we didn’t get any matches with labs, so I presumed that the labs weren’t part of ROR. Interesting to know that that’s not necessarily the case.

Amanda French

Right. So I’m looking randomly, but here’s the Laboratory of Astrophysics of Bordeaux, for instance.

Allyson Lister

Ah, yes, I see. So they are there, and they have the right parent organization. I mean, we’d be very happy to pull ROR IDs for laboratories. For example, the He Group: they are a really prolific ontology development group, and they’re heavily involved with the OBO Foundry which is a big life science and biomedical ontology community, and so they have loads of records. But that wasn’t a match in ROR. Maybe because they never submitted themselves? I don’t know. But we’d be very happy to have a ROR link for things like that. And those are the sorts of things that didn’t come up with matches. Whether or not they actually are in scope, and we just need better communication, perhaps.

Amanda French

That was the other thing I was going to ask, is whether you or anyone on the FAIRsharing team ever says, “Oh, let’s add this to ROR.” Not that that’s your job!

Allyson Lister

We have thought about it, and I think once or twice I did use the form. But it’s time, isn’t it. And when you are, as you and we both are, manually curated resources, you always have a pinch point of curation, which is one of the reasons we are building community curators. And no matter how hard you try, it’s difficult to keep up. Now, we have a commitment that we are trying to look at every record at least once a year, and together with our maintainers, who are meant to look at their records once a year, and the community curators, that’s our goal.

Amanda French

Right.

Allyson Lister

So I know that in that sense it’s hard for us, so if we don’t find the match but we’re not sure, or if we do find a match but there are metadata fields that don’t match, we tend to just keep ours, but add the ROR link. It could be that we have a more up to date web page, for instance, or sometimes it’s the other way around. And so that’s why we haven’t sort of en masse shunted all the metadata fields from ROR to FAIRsharing, because that would require another check. But there’s no reason why we couldn’t investigate sharing of some kind of the results of these three-monthly runs in sort of a more automated fashion. So if you had time to look at what we found and see where the differences are, we’d give you a list of differences. But I think for that we’d have to talk a little bit more about an official collaboration so we could say “This is what the goals are, this is what the end result will be.” Then something like the output of some of these regular runs might inform the curation you do, rather than having to go and fill out the form every time.

Amanda French

Oh, certainly, and our curation process is also new, and we have a nice community curation process as well, but we have a large volume of requests, even with just that single form, right. So we are doing wonderfully right now, but I don’t know that we have the capacity to do a lot of things. However, we have come across a number of these projects where they do have data about organizations. We do know that we want to examine or analyze it, but we just don’t have the capacity to do it. So you know, that does suggest a grant-funded project. But we are continuing to work on building a curation process that is really robust and scalable, and it would be great to try to create a sort of an abstractable way to do this with anyone who is adopting ROR. Because almost anybody who is adopting ROR has this issue. They have data about things that are not in ROR – maybe because they shouldn’t be, but maybe they should be. So we’re sort of just beginning to look at that. But we have one Curation Lead and something like half a dozen community curators, so we’re not looking for work right now. And I have to say we’ve had something like 1,500 curation requests this year. We get requests all the time, and they have to be evaluated and curated. We are proud of that, too, like you are, that these are well-curated records. We haven’t touched all of them, certainly, but we are doing a lot of curation. [*Note: in 2022 ROR added 2,010 new records and updated 10,073 records – about 10% of the registry – from external curation requests and internal quality assurance.]

Allyson Lister

You don’t have to explain to me. It’s the same issue with any curated resource, I think. My first job was with Swiss-Prot and TrEMBL; they’re now called UniProt, which is a protein sequence database. You’re coming from the library sciences, so you may not have come across them, but I was a biologist first, and then I moved into data science and I was a database programmer for them in Cambridge. Not the university, but an institute there. And they had a team of, probably, between the main group of curators in Cambridge and another group in Geneva, forty or fifty Ph.D.-level curators that were manually curating, and they could spend more than a week on one record, because they’d read all the publication material and could do extremely thorough curation. And there was simply no way they were going to keep up with all protein sequence submissions. A manually-curated resource is an amazing resource, but you’re always going to be fighting against scarcity.

Amanda French

I’ve done some work with, you know, crowdsourcing types of projects, and the ideal is to automate as much as you can, and to build tools that help the manual curators. If you have some kind of continuum between “everything is manually curated” and “everything is automatic,” my ideal for good quality metadata is that you automate as much as you can, and you build tools to help the human curators, and that produces to my mind the very best metadata. If you do only one or only the other, humans make mistakes, machines are very dumb and make mistakes, so you know, you really have to find that hybrid model that works well for everybody.

Allyson Lister

It’s true, because when we’re curating a resource, we have to look at all these different types of websites with all these different types of things. We can’t really screen scrape; it’s not really been an option. But with the advent of things like Bioschemas and schema.org, that kind of thing might happen in the future, but it’s not there yet. Certainly for our integrations, we look to automate as much as we can with checks, with curations like the ROR integration for organizations. Now is such an exciting time, because when we made the new data model, we wanted to think of an organization as a digital object. Before that it was the record, the description of the resource that was at the center of everything in FAIRsharing, and that’s still important to us. But we’re broadening our idea of what’s a node in our graph, if you see what I’m saying. There’s a graph for every single record in FAIRsharing. So you go to the Dryad record, you go to the ROR record, and you look at the graph, and it shows you all of its relationships that it has with all of the other resources that implement it, or the databases that share data with it. Hidden and not displayed yet, but still there in a graph, are the organizations, because the organizations themselves have their own digital object in FAIRsharing. And our next iteration of the graph will showcase that, so not only will you be able to see the resource landscape, you’ll see the resource landscape integrated with the organizations that develop and maintain and fund it. And if you look at our organization pages now, like the one for GBIF, you can see that we’ve put all the information there on its page.

That’s our GBIF record. You can see that they sit as part of a larger ecosystem. So GBIF has seventeen records that it’s related to in some way, that it maintains, funds, anything. And it has its ROR ID and it has its country, because a lot of people like to find resources built in certain areas, and it has the users that are maintainers or community curators. In this case, there are community curators who are affiliated with the organization to broaden the scope of that graph. And then what you get from that is the ability to represent organizations as first-class citizens in FAIRsharing through relationships with people like ROR, and through the links they have to their resources, which are described in FAIRsharing. And our GBIF community curator Kyle Copas has particularly focused on the organizations within his consortium.

And so GBIF said, “Well, we are part of this BiCIKL consortium, and we need to find out, for instance, all the resources that are associated with our consortium members and if we’re missing anything. So how can we build a graph?” And Kyle said, “I know! FAIRsharing has graphs. But they’re not complete yet.” And he came to us and he became an early adopter. This was right at the very beginning of our community curator program. He’s one of the very first. And he used his expert knowledge to enrich the curation on our records related to his consortium. For him, it was the records and the relationships they had with their organizations that was most interesting. This is where it all ties back to ROR, and organizations as important members of an ecosystem of resources. As a community curator, he received attribution for his role with us on his ORCID profile.