Based on DOIs from the following file / list: {{ currentGraph.tabLabel }} (link to share)

Data retrieved through {{ currentGraph.API }} API on {{ new Date(currentGraph.timestamp).toLocaleDateString() }} (estimated completeness: {{ completenessPercent }}%).

{{ inputArticlesFiltered.length + " / " + currentGraph.input.length }} {{ props.row.numberInSourceReferences }} {{ props.row.isSource ? 'Source: ' : '' }} {{ props.row.authors[0].LN }} {{ props.row.year }}

Incoming suggestions are the locally most cited references from the input articles. They are generally older. Maximum:

{{ incomingSuggestionsFiltered.length + " / " + maxIncomingSuggestions }} {{ props.row.authors[0].LN }} {{ props.row.year }}

Outgoing suggestions are citing the most input articles. They are generally newer. Maximum:

{{ outgoingSuggestionsFiltered.length + " / " + maxOutgoingSuggestions }} {{ props.row.authors[0].LN }} {{ props.row.year }}
({{ (fullscreenNetwork) ? 'show tables' : 'fullscreen network' }}) ↑ newer articles (how to read this network) ↓ older articles
({{ (fullscreenNetwork) ? 'show tables' : 'fullscreen network' }}) (how to read this network)

Click + to enter source DOI
or scan a text file for DOIs instead !

(Load examples instead or see Frequently asked questions)


What is this web app about?

This web app aims to help scientists with their literature review using metadata from Microsoft Academic, Crossref and OpenCitations. Academic papers cite one another, thus creating a citation network (= graph). Each node (= vertex) represents an article and each edge (= link / arrow) represents a reference / citation. Citation graphs are a topic of bibliometrics, for which other great software exists as well.

This web app visualizes subsets of the global citation network that I call "local citation networks", defined by the references of a given set of input articles. In addition, the locally most cited references missing in the set of input articles are suggested for further review.

How to use this web app?

There are basically two ways to create new networks: First, based on the references of a given paper. This could be a review or just an interesting paper you'd like to dig into deeper. Either click the + button at the top right and enter the digital object identifier (DOI) of the source article or use the bookmarklet described below.

The second way to create a new network is to scan a text file for DOIs, which allows you to create your own unique lists of input articles. For example, this could be a .ris / .bib / .json file exported by your reference manager (e.g. Zotero or EndNote). In theory, you can scan any plain text file (PDFs and Microsoft Office files don't work). However, due to the multitude of characters allowed in DOIs, the system might not catch all of them correctly (see this blog post by Crossref for more explanation).

Up to 5 tabs can be open at the same time, representing a different local citation network each. Only one query can be processed at a time, which is why some buttons are disabled when a query is still going on. Microsoft Academic seems to be able to process quite large numbers of references (I've successfully tried up to 500) but Crossref's API returns an error when trying to process too many references (depending on the length of the search string, around >110 DOIs).

How to use the bookmarklet to open references of a paper?

A bookmarklet is a small program that can be saved as a bookmark in your browser. This particular bookmarklet allows you to open a new network in Local Citation Network directly when surfing many journals' websites (e.g. Nature, Science, Lancet, ...) instead of having to enter the DOI manually. Save the following link as a bookmark (e.g. by dragging it to your bookmarks bar or by right-click β†’ "Bookmark This Link" in Firefox) and then click on it when reading an article (e.g. this one): Open in Local Citation Network.

The bookmarklet is particularly useful when you want to use Local Citation Network as a "second screen" when you are reading a paper, showing you its references in the correct order and allowing you to view their abstracts and to identify seminal articles as well as additional suggested articles. When using the bookmarklet, the reference list is not obtained through Microsoft Academic itself but scanned directly from the website and then only subsequently this list of references is sent to Microsoft Academic to construct the Citation Network and to obtain suggestions. This way, the original order of references is preserved, which is not the case when using the Microsoft Academic API.

Unfortunately, not all journals' websites are supported. If you know how to code, check out the code of the bookmarklet on GitHub and extend it to support more journals! In this light it might be worth updating the bookmark in your browser from time to time to benefit from possible improvements.

Where do the 'Incoming suggestions' come from?

'Incoming suggestions' are the locally most cited references from the input articles, thus they have the highest in-degree. In order to create the local citation network, this web apps attempts to retrieve reference-lists from all input articles (usually some are missed, see completeness) and then orders these references by the number of local incoming citations (in-degree, see network). Some of these references might be among the input articles, depending on how connected they are. However, some of them usually aren't among the input articles and then become 'incoming suggestions' (they must be referenced by at least two input articles). Currently the arbitrary maximum number of incoming suggestions is 20 in order to avoid API overload.

Keep in mind that because of the nature of this approach, 'incoming suggestions' are generally older than the input articles. This can help identifying seminal papers of the field but not newer state-of-the-art research. In order to identify newer articles, use OpenCitations to obtain outgoing suggestions.

Where do the 'Outgoing suggestions' come from?

'Outgoing suggestions' are citing the most input articles, thus they have the highest out-degree. They are generally newer than the input articles. Only the OpenCitations API serves a field 'citation', which is why it's the only API supporting this feature. Outgoing suggestions must cite at least 2 input articles and currently their arbitrary maximum is 20 in order to avoid API overload.

How to read the citation network?

Zoom by scrolling and pan by dragging. Each node (= vertex) represents an article. They are ordered by year from top (newer) to bottom (older), with each year colored differently. The size of the nodes depends on the sum of their "in-degree", the number of citations the paper received from the input articles, and their 'out-degree', the number of input articles they cite. The local 'in-degree' is not always proportional to the global citation count. Input articles are circle-shaped nodes ( ● ), except for the source article, which is diamond-shaped (β—†), if applicable. Up-pointing triangles (β–²) represent incoming suggestions, which are cited by the input articles but not part of them themselves. Down-pointing triangles (β–Ό) are outgoing suggestions, which are citing the input articles but not part of them themselves.

Each edge (= link / arrow) represents a reference and citation at the same time: They are references from their outgoing nodes and citations for their incoming nodes. Usually papers can only cite older papers, hence edges tend to point downwards (or sidewards for citations of the same year). Technically this is called a directed acyclic graph.

How to read the co-authorship network?

This "local" author network shows the commonest authors among the source, input and suggested articles. Depending on the discipline, the number of (co)authors among a set of multiple articles can quickly rise to the hundreds, which is why this author network only shows authors with a minimum number of local articles (not to be mistaken with their global number of publications, which is usually much higher). The default minimum number of local articles is determined so that this network shows no more than 50 authors, but it can be changed with the slider at the bottom from 2 to 10.

Each node (= vertex) represents an author. The size depends on their number of local articles and the color depends on the author-cluster they first appear in. Diamond-shaped (β—†) nodes are also authors of the source article whose references define the input articles. Click authors to filter input & suggested articles, ctrl+click to filter for more than one author.

Each edge (= link) represents a collaboration between two authors among the source, input and suggested articles, the width indicating the number of collaborations. Click edges to filter input & suggested articles.

Which API should be used and how to switch?

Microsoft Academic (MA) is used by default because it is usually the most comprehensive and also includes abstracts. Switch the API with the button indicating MA / CR / OC in the top-right menu. See background for more details on the APIs used.

Crossref (CR) has the advantage that it usually returns reference lists in their original order (alternatively you can use the bookmarklet). Also, it allows for the most accurate estimation of the reference-list completeness, it's usually very fast and can handle fairly large input lists.

OpenCitations (OC) is the only API featuring outgoing suggestions, which are newer than the input articles. However, it can be quite slow and often returns an error for larger input lists.

What does 'Autosave results locally' do?

When activated, this option caches your recent networks (up to 5 tabs) and your settings (including your personal Microsoft Academic API key) locally in your browser in the so-called localStorage. When you revisit this web app, you can pick up where you left. When deactivated, this data is deleted.


This web app is not working!? Which browsers are supported?

I've tested with current versions of Firefox, Safari and Chrome (as of November 2019) and made sure this web app runs in these. Microsoft Edge and Microsoft Internet Explorer are unfortunately not supported and do not work at all right now. This web app is meant for desktop use and not optimized for smartphones, which have not been tested at all. I'm a single person and created this web app in my spare time for free and for fun, so I don't have the time and resources to test all possible devices, web browsers and scenarios. This web app is open source, feel free to support it by extending compatibility or optimizing mobile / tablet use! If you find a bug or problem with the newest versions of Firefox / Safari / Chrome, please report it on GitHub or contact me.

The queries have turned very slow!? / How to cancel loading?

Both the Microsoft Academic API and the Crossref API are public with varying response speeds depending on their overall usage and workload. In addition, I hypothesize they might be throttling individual users (as identified by IP address) when they perform too many queries in a row. Checking your internet connection, only querying what's really important, switching to the other API and trying again later might help!

Sometimes (particularly with unstable internet connections) queries seem to run forever. In case you want to cancel loading, activate 'Autosave results locally' (see autosave) and refresh page. You can deactivate autosave afterwards again.

Is the data complete? / Some references are missing!?

The data is usually incomplete. Some DOIs cannot be found at all in Microsoft Academic / Crossref / OpenCitations and for those that are found, often some metadata like authors, abstracts or reference-lists are missing (particularly in Crossref and OpenCitations). For the citation network and incoming suggestions, the completeness of the reference-lists is most important. Usually Microsoft Academic's data is more complete, which is currently turned {{ (API === 'Microsoft Academic') ? 'on' : 'off' }} (how to toggle).

The estimated completeness can be seen above the search bar in the "Input articles" tab and is calculated in the following way: For Microsoft Academic & OpenCitations it is the fraction of input articles that have reference-lists themselves (multiplied by the fraction of specified DOIs found, in case of an input list).

Crossref allows more subtle calculation, as it often also provides the total reference count, which is often larger than the reference-list of DOIs (older references often don't have DOIs and neither do some specific books, papers or conference abstracts). The estimated completeness is thus calculated as the product of three fractions:

  1. Source reference completeness: (Number of input articles found in Crossref) / (Total reference count of source or total number of DOIs in file)
  2. (Number of input articles that have reference-lists themselves) / (Number of input articles)
  3. Average input articles' reference completeness among those input articles that do have reference-lists: (Length of reference-list) / (Total reference count)

How can I preserve the original order of references in the input article list?

Unfortunately, Microsoft Academic does not seem to save the original order in which references appear in an article, at least it does not return it via its API. However, you can use the bookmarklet to open reference lists in their original order.

Alternatively, you can use Crossref, which does save the original order of references. See here how to switch the API used.


What is bibliometrics? What other software can you recommend?

Bibliometrics is a part of information science that uses statistical methods to analyze publications. It emcompasses scientometrics, which is specifically looking at scientific publications, and citation analysis.

A highly recommended tool similar in spirit to this one is Citation Gecko (based on Crossref and OpenCitations), which I didn't know when creating Local Citation Network. The main difference is that you start with a small number of seed articles (5-6 are recommended) and incrementally increase the network by adding recommended articles to the pool of seed articles. Recommended articles include all references and even global citations. Check out this guest post I've written for the blog Leiden Madtrics comparing the two tools.

Other great software to help with bibliometric / scientometric analyses include Anne-Wil Harzing's Publish or Perish, which allows sophisticated queries of a plethora of APIs including Microsoft Academic and Crossref but also Google Scholar, Scopus and Web of Science. If you're looking to create more complex networks than this web app allows, check out the Leiden University's great VOSviewer and CitNetExplorer, which allow you to create and customize everything from citation networks to author networks, university networks and journal networks. VOSviewer also allows text mining and co-occurrence networks.

Where is the data coming from, which API is used?

Local Citation Network supports APIs from Microsoft Academic, Crossref and OpenCitations. Microsoft Academic is the default as it is most comprehensive and also includes abstracts. It's a freely available academic search-engine based on the proprietory Microsoft Academic Graph (similar to Clarivate's Web of Science and Elsevier's Scopus, both of which require (paid) subscriptions). Several publications have analyzed the quality and coverage of Microsoft Academic, with overall very promising results. Hug et al. (2017) have said: "The citation analyses in MA [Microsoft Academic] and Scopus yield uniform results". Microsoft Academic's API needs a personalized key to use, which is free for up to 10,000 transactions per month. This web app comes with a test key, but it's faster and more reliable to get your own free key here!

Crossref is a not-for-profit organization that collects metadata from scholarly journals and publications. OpenCitations is an independent infrastructure organization extending on these ideas, not just providing (outgoing) reference-lists, but also (incoming) citations for each indexed article. Thanks to the formidable Initiative for Open Citations (I4OC), the coverage of Crossref and OpenCitations has increased vastly. However, some journals still do not support these ideals - feel free to write them, in case you find one! Crossref's API and OpenCitations' API are freely available.

Why aren't you using Google Scholar or Semantic Scholar's API?

Unfortunately Google Scholar doesn't share their data and discourages webscraping. Consequently, it doesn't have an API.

Semantic Scholar has a free API, which is great. Unfortunately however, it is too limited: It doesn't allow to retrieve information on multiple DOIs at once and enforces a rate limit, effectively rendering it unsuitable for this project's usecase.

I have an idea! Can you implement XYZ?

It's great to see you're getting involved! While I see numerous ways to extend this project myself, I don't have much time to work on it anymore unfortunately. However, this web app is open source and I'd be very happy to see others extend this project! In the meantime, check out these great applications, which might already provide what you're looking for.

I'd like to contribute! Is this open source and are you looking for help? / Is this on GitHub?

Yes, great! This project is open source (GPL-3) and the source can be found on GitHub. I don't have much more time to work on this project, so I'm actively looking for help and contributors. Bugfixes are always welcome but please contact me before any large pull-requests so we can coordinate.

Does this web app use cookies? What's the privacy policy?

No, this web app doesn't use any cookies. No user-tracking or fingerprinting is performed and no usage-data is collected by the author of this web app. Optionally, so-called localStorage can be used to cache results and settings locally when "Autosave results locally" is manually turned on (see autosave).

This web app is running locally on your computer and only contacts the API servers of Microsoft Academic / Crossref during runtime. Microsoft has a general privacy statement, I haven't found anything specific to the Microsoft Academic API. Crossref also has a general privacy statement. This web app is hosted by GitHub pages, see their privacy statement here.

Technicalities: Which JavaScript libraries are used and why no CDN?

This web app uses Vue.js, Buefy and vis.js Network. I opted against using content delivery networks (CDN) to speed up delivery of these JavaScript libraries and thus initial loading-time, because of the potential user-tracking by CDNs. If you're interested, this rising problem is partly countered by the great browser-plugin Decentraleyes (see their FAQ for more details).

Who are you, why did you create this and how can I contact you?

I'm Tim WΓΆlfle from Germany and started creating this project during the literature review for my Master's thesis. I've often manually scanned reference lists of papers to identify seminal articles and thought this must be automized. You can contact me on Twitter or by e-mail.