Choosing the right tools for measuring the online impact of research

Researchers are increasingly using non-academic platforms to amplify the impact of their findings among the general public. These efforts require measurement to assess strategy effectiveness to increase research visibility, guide future research directions, and demonstrate relevance to funders and policymakers.

Traditional citation-based metrics offer a limited perspective on impact, failing to capture discussions generated on platforms where research is informally cited or presented in formats different to scientific articles.

To address this gap, Jason Priem coined the term “altmetrics” in 2010. These complementary indicators reflect the attention individual research outputs (e.g., articles, conference proceedings, book chapters) receive from various online sources. In 2011, Priem co-authored the Altmetrics Manifesto with Dario Taraborelli, Paul Groth, and Cameron Neylon, outlining the core principles and potential of these new metrics.

Three main platforms now provide summaries of the extent, manner, and locations in which academic work is discussed online:

Altmetric.com

Altmetric.com is a platform, founded by Euan Adie in 2011, that collects information from three main sources: social media, traditional media, and online reference managers. It is recognized for providing an attention score at the center of a donut whose colors reflect the sources of attention a research output received.

The attention score is a weighted count of all attention the research received in terms of: 

  • Volume: the amount of people that mention a particular research output. The score rises as more people mention it. Almetric only counts one mention per person per source. 
  • Sources: each category of mention has a different weight. For instance, a newspaper article provides more points than a blog, and a blog´s weight is bigger than that of a tweet. 
  • Authors: how often the author of each mention talks about scholarly articles. 

Altmetrics is the leading provider of altmetrics both because it was one of the pioneering companies that emerged shortly after the altmetrics concept was introduced, and also because of its user-friendly interface and as well as its powerful filters. Institutions like Cambridge University, University of Michigan, Schulich School of Medicine and Dentistry, University of Essex, Queensland University of Technology, and more, are currently using this tool. 

Plum Analytics 

Plum Analytics was also founded in 2011. Six years later, in 2017, it was purchased by Elsevier. Plum X Metrics is the tool offered by Plum Analytics to gather the metrics of sixtyseven types of research artifacts, including less common research outputs such as conference papers, newsletters, maps, manuscripts and interviews. 

PlumX Metrics provides an analysis segmented into five categories: 

  • Citations: contains both traditional citation indexes such as Scopus, as well as citations that help indicate societal impact like Clinical or Policy Citations.
  • Usage: informs whether anyone is reading the articles or otherwise using the research, by measuring clicks, downloads, views, library holdings, video plays, and collaborators. 
  • Captures: indicates that someone wants to come back to the work (bookmarks, favorites, followers, forks, readers, exports/saves, subscribers, and watchers).
  • Mentions: measures activities such as news articles or blog posts about the research.
  • Social media: is a good measure of how well a piece of research has been promoted. Includes shares, likes, etc.

The company has worked with Georgia Southern University, The University of Rome Tor Vergata, Autism Speaks, Pacifica Graduate Institute, and Texas A&M, among others.

 

Crossref

Crossref, founded in 2000 by a group of scholarly publishers, offers a non-profit open source tool that measures altmetrics. It is designed mainly for organizations, including publishers, research institutions, universities, funders, museums, government organizations, libraries, data and subject repositories, conference providers, standards bodies, and news outlets that want to create a lasting and reusable scholarly record. 

Member institutions register their content (journals, books, book chapters, datasets, dissertations, reports, peer reviews, grants, and others) using metadata (data that provides information or describes other data). Basic metadata includes titles, author names, ISSNs/ISBNs, abstracts, references, DOI and URL of content location. In depth information on all the processes involving becoming a member can be found in this recent webinar. 

Altmetrics offer researchers powerful tools to measure the online impact of their work beyond traditional metrics. As the digital landscape continues to evolve, utilizing these alternative metrics becomes increasingly important for maximizing the visibility and influence of your scholarly contributions.

More on Decilion Insight

Top five data visualization tools for health researchers
Public health and medical research on social media: opportunity or mismatch?

More on Decilion Insight