Based on DOIs from the following file / list: {{ currentGraph.tabLabel }} (link to share)

Data retrieved through {{ currentGraph.API + ' (' + abbreviateAPI(currentGraph.API) + ')' }} API on {{ new Date(currentGraph.timestamp).toLocaleDateString() }} (estimated completeness: {{ completenessPercent }}%).

{{ inputArticlesFiltered.length + " / " + currentGraph.input.length }} {{ props.row.numberInSourceReferences }} {{ props.row.isSource ? 'Source: ' : '' }} {{ props.row.title }} {{ props.row.authors[0] && props.row.authors[0].LN }}

Incoming suggestions are the locally most cited references from the input articles, thus they have high in-degrees. They tend to be older than most input articles. Number:

{{ incomingSuggestionsFiltered.length + " / " + maxIncomingSuggestions }} {{ props.row.title }} {{ props.row.authors[0] && props.row.authors[0].LN }}

Outgoing suggestions are citing the most input articles, thus they have high out-degrees. They tend to be newer than most input articles. Number:

{{ outgoingSuggestionsFiltered.length + " / " + maxOutgoingSuggestions }} {{ props.row.title }} {{ props.row.authors[0] && props.row.authors[0].LN }}
({{ (fullscreenNetwork) ? 'show tables' : 'fullscreen network' }}) (how to read this network) (how to read this network)
({{ (fullscreenNetwork) ? 'show tables' : 'fullscreen network' }}) (how to read this network)

Click + to enter source DOI
or scan a text file for DOIs instead !

(Load examples instead or see Frequently asked questions)


What is this web app about?

This web app aims to help scientists with their literature review using metadata from OpenAlex (OA), Semantic Scholar (S2) and Crossref (CR). Academic papers cite one another, thus creating a citation network (= graph). Each node (= vertex) represents an article and each edge (= link / arrow) represents a reference / citation. Citation graphs are a topic of bibliometrics, for which other great software exists as well.

This web app visualizes subsets of the global citation network that I call 'local citation networks', defined by the references (and citations) of a given set of input articles. In addition, the locally most relevant references (and citations) missing in the set of input articles are suggested for further review.

How to use this web app?

There are basically two ways to create new networks: First, based on the references of a given paper. This could be a review or just an interesting paper you'd like to dig into deeper. Either click the + button at the top right and enter the digital object identifier (DOI) of the source article or use the bookmarklet described below.

The second way to create a new network is to scan a text file for DOIs, which allows you to create your own unique lists of input articles. For example, this could be a .ris / .bib / .json file exported by your reference manager (e.g. Zotero or EndNote). In theory, you can scan any plain text file (PDFs and Microsoft Office files don't work). However, due to the multitude of characters allowed in DOIs, the system might not catch all of them correctly (see this blog post by Crossref for more explanation).

Up to 5 tabs can be open at the same time, representing a different local citation network each. Only one query can be processed at a time, which is why some buttons are disabled when a query is still going on. The APIs can return errors when they're overloaded, the reference list is too long or your search frequency too high.

How to use the bookmarklet to open references of a paper?

A bookmarklet is a small program that can be saved as a bookmark in your browser. This particular bookmarklet allows you to open a new network in Local Citation Network directly when surfing many journals' websites (e.g. Nature, Science, Lancet, …) instead of having to enter the DOI manually. Save the following link as a bookmark (e.g. by dragging it to your bookmarks bar or by right-click → 'Bookmark Link' in Firefox) and then click on it when reading an article (e.g. this one): Open in Local Citation Network.

The bookmarklet is particularly useful when you want to use Local Citation Network as a 'second screen' when you are reading a paper, showing you its references in the correct order and allowing you to view their abstracts and to identify seminal articles as well as additional suggested articles. When using the bookmarklet, the reference list is scanned directly from the website and then only subsequently this list of references is sent to the selected API to construct the Citation Network and to obtain suggestions. This way, the original order of references is preserved.

Unfortunately, not all journals' websites are supported. If you know how to code, check out the code of the bookmarklet on GitHub and extend it to support more journals! In this light it might be worth updating the bookmark in your browser from time to time to benefit from possible improvements.

Where do the 'Incoming suggestions' come from?

'Incoming suggestions' are the locally most cited references from the input articles, thus they have high in-degrees. In order to create the local citation network, this web apps attempts to retrieve reference-lists from all input articles (usually some are missed, see completeness) and then orders these references by the number of local incoming citations (in-degree, see network). Some of these references might be among the input articles, depending on how connected they are. However, some of them usually aren't among the input articles and then become 'incoming suggestions' (they must be referenced by at least two input articles). Currently the arbitrary maximum number of incoming suggestions is 20 in order to avoid API overload.

Keep in mind that because of the nature of this approach, 'incoming suggestions' are generally older than the input articles. This can help identifying seminal papers of the field but not newer state-of-the-art research. In order to identify newer articles, use Semantic Scholar to obtain outgoing suggestions.

Where do the 'Outgoing suggestions' come from?

'Outgoing suggestions' are citing the most input articles, thus they have high out-degrees. They are generally newer than the input articles. Local Citation Network currently only supports this feature with the Semantic Scholar API. Outgoing suggestions must cite at least 2 input articles and currently their arbitrary maximum is 20 in order to avoid API overload.

How to read the citation network?

Zoom by scrolling and pan by dragging. Each node (= vertex) represents an article. They are ordered by year from top (newer) to bottom (older), with each year colored differently. The size of the nodes depends on the sum of their 'in-degree', the number of citations the paper received from the input articles, and their 'out-degree', the number of input articles they cite. The local 'in-degree' is not always proportional to the global citation count. Input articles are circle-shaped nodes ( ● ), except for the source article, which is diamond-shaped (◆), if applicable. Up-pointing triangles (▲) represent incoming suggestions, which are cited by the input articles but not part of them themselves. Down-pointing triangles (▼) are outgoing suggestions, which are citing the input articles but not part of them themselves (currently only available with the Semantic Scholar API).

Each edge (= link / arrow) represents a reference and citation at the same time: They are references from their outgoing nodes and citations for their incoming nodes. Usually papers can only cite older papers, hence edges tend to point downwards (or sidewards for citations of the same year). Technically this is called a directed acyclic graph.

How to read the co-authorship network?

This 'local' author network shows the commonest authors among the source, input and suggested articles. Depending on the discipline, the number of (co)authors among a set of multiple articles can quickly rise to the hundreds, which is why this author network only shows authors with a minimum number of local articles (not to be mistaken with their global number of publications, which is usually much higher). The default minimum number of local articles is determined so that this network shows no more than 50 authors, but it can be changed with the slider at the bottom from 2 to 10.

Each node (= vertex) represents an author. The size depends on their number of local articles and the color depends on the author-cluster they first appear in. Diamond-shaped (◆) nodes are also authors of the source article whose references define the input articles. Click authors to filter input & suggested articles, ctrl+click to filter for more than one author.

Each edge (= link) represents a collaboration between two authors among the source, input and suggested articles, the width indicating the number of collaborations. Click edges to filter input & suggested articles.

Which APIs are supported and how to switch?

Switch the API with the button indicating "OA / S2 / CR" in the top-right menu.

OpenAlex (OA) is used as the default as the direct successor of Microsoft Academic, which was discontinued in 01/2022. It is fast, it has high coverage and serves abstracts. It also serves a flag for retracted papers, which can be seen here for example (it can be missing though).

Semantic Scholar (S2) also has high coverage, also serves abstracts and allows accurate estimation of the reference-list completeness. In addition, it easily serves incoming citations for each paper, allowing the generation of outgoing suggestions, which are newer than the input articles. However, it can be slow and has a rate limit of 100 requests (i.e. papers) per 5 minutes per user, limiting its use.

Crossref (CR) has the advantage that it usually returns reference lists in their original order (alternatively you can use the bookmarklet). Also, it allows for an accurate estimation of the reference-list completeness, it's usually very fast and can handle fairly large input lists. Crossref is a not-for-profit organization that collects metadata from scholarly journals and publications. Thanks to the formidable Initiative for Open Citations (I4OC), the coverage of Crossref has increased vastly. However, some journals still do not support these ideals - feel free to write them, in case you find one!

The metadata API from OpenCitations (OC) theoretically also features outgoing suggestions. However, it can be slow and unstable, which is why it is deactivated by default. Activate it here experimentally, but I advise to only try very small queries.

What does 'Autosave results locally' do?

When activated, this option caches your recent networks (up to 5 tabs) and your preferred API locally in your browser in the so-called localStorage. When you revisit this web app, you can pick up where you left. When deactivated, this data is deleted. See privacy.


This web app is not working!? Which browsers are supported?

I've tested with the current version of Firefox (as of February 2022), but this web app should also run in current versions of Safari, Edge and Chrome. Microsoft Internet Explorer is not supported. This web app is meant for desktop use and not optimized for smartphones, which have only been tested sparsely. I'm a single developer and created this web app in my spare time for free and for fun, so I don't have the time and resources to test all possible devices, web browsers and scenarios. This web app is open source, feel free to support it by extending compatibility or optimizing mobile / tablet use! If you find a bug or problem with the newest versions of Firefox / Safari / Edge / Chrome, please report it on GitHub or contact me.

The queries have turned very slow!? / How to cancel loading?

APIs are public with varying response speeds depending on their overall usage and workload. In addition, I hypothesize they might be throttling individual users (as identified by IP address) when they perform too many queries in a row. Checking your internet connection, only querying what's really important, switching to the other API and trying again later might help!

Sometimes (particularly with unstable internet connections) queries seem to run forever. In case you want to cancel loading, activate 'Autosave results locally' (see autosave) and refresh page. You can deactivate autosave afterwards again.

Is the data complete? / Some references are missing!?

The data is usually incomplete. Some DOIs cannot be found at all in some APIs and for those that are found, often some metadata like authors, abstracts or reference-lists are missing. For the citation network and incoming suggestions, the completeness of the reference-lists is most important.

The estimated completeness of the reference-lists can be seen above the search bar in the "Input articles" tab and is calculated in the following way: For OpenAlex and OpenCitations it is the fraction of input articles that have reference-lists themselves (multiplied by the fraction of specified DOIs found, in case of an input list).

Semantic Scholar and Crossref allow for more subtle calculation, as they often also provide the total reference count, which is often larger than the reference-list of DOIs (older references often don't have DOIs and neither do some specific books, papers or conference abstracts). The estimated completeness is thus calculated as the product of three fractions:

  1. Source reference completeness: (Number of input articles found in API) / (Total reference count of source or total number of DOIs in file)
  2. (Number of input articles that have reference-lists themselves) / (Number of input articles)
  3. Average input articles' reference completeness among those input articles that do have reference-lists: (Length of reference-list) / (Total reference count)

How can I preserve the original order of references in the input article list?

You can use the bookmarklet to open reference lists in their original order. Alternatively, you can use Crossref, which does save the original order of references. See here how to switch the API used.


What is bibliometrics? What other software can you recommend?

Bibliometrics is a part of information science that uses statistical methods to analyze publications. It emcompasses scientometrics, which is specifically looking at scientific publications, and citation analysis.

The first tool similar in spirit to this one was Citation Gecko (based on Crossref and OpenCitations), which I didn't know when creating Local Citation Network. The main difference is that you start with a small number of seed articles (5-6 are recommended) and incrementally increase the network by adding recommended articles to the pool of seed articles. Recommended articles include all references and even global citations. Check out this guest post I've written for the blog Leiden Madtrics in 12/2019 comparing the two tools.

Lately, newer similar tools have emerged like Connected Papers, Litmaps and Inciteful. Aaron Tay has written a blog post in 06/2021 for comparison.

Other great software to help with bibliometric / scientometric analyses include Anne-Wil Harzing's Publish or Perish, which allows sophisticated queries of a plethora of APIs including Crossref but also Google Scholar, Scopus and Web of Science. If you're looking to create more complex networks than this web app allows, check out the Leiden University's great VOSviewer and CitNetExplorer, which allow you to create and customize everything from citation networks to author networks, university networks and journal networks. VOSviewer also allows text mining and co-occurrence networks.

Why aren't you using Google Scholar?

Unfortunately Google Scholar doesn't share their data and discourages webscraping. It doesn't have an API.

I have an idea! Can you implement XYZ?

It's great to see you're getting involved! While I see numerous ways to extend this project myself, I don't have much time to work on it anymore unfortunately. However, this web app is open source and I'd be very happy to see others extend this project! In the meantime, check out these great applications, which might already provide what you're looking for.

I'd like to contribute! Is this open source and are you looking for help? / Is this on GitHub?

Yes, great! This project is open source (GPL-3) and the source can be found on GitHub. I don't have much more time to work on this project, so I'm actively looking for help and contributors. Bugfixes are always welcome but please contact me before any large pull-requests so we can coordinate.

Does this web app use cookies? What's the privacy policy?

No, this web app doesn't use any cookies. No user-tracking or fingerprinting is performed and no usage-data is collected by the author of this web app. Optionally, so-called localStorage can be used to cache results and settings locally when "Autosave results locally" is manually turned on (see autosave).

This web app is running locally on your computer and only contacts the API servers during runtime. Crossref has a general privacy statement. This web app is hosted by GitHub pages, see their privacy statement here.

Technicalities: Which JavaScript libraries are used and why no CDN?

This web app uses Vue.js, Buefy and vis.js Network. I opted against using content delivery networks (CDN) to speed up delivery of these JavaScript libraries and thus initial loading-time, because of the potential user-tracking by CDNs. If you're interested, this rising problem is partly countered by the great browser-plugin Decentraleyes (see their FAQ for more details).

Who are you, why did you create this and how can I contact you?

I'm Tim Wölfle from Germany and started creating this project during the literature review for my Master's thesis. I've often manually scanned reference lists of papers to identify seminal articles and thought this must be automized. You can contact me on Twitter or by e-mail.