About Internet Health Report (IHR)

The Internet Health Report monitors the conditions of networks that compose the Internet. It aims to provide network operators, policymakers, and other stakeholders, with a better understanding of the Internet's infrastructure and its evolution. Network data is obtained from public large-scale measurement platforms, such as RIPE Atlas, RIS, and Route Views. This data is analyzed with a set of tools pinpointing delay changes, forwarding anomalies, network disconnections and routing changes. In a general effort towards transparency and reusability, the Internet Health Report data and analysis tools are made publicly available.

Monitored networks

For delay changes, forwarding anomalies, network disconnections, the data we analyze is collected with RIPE Atlas. Networks observed in the builtin and anchoring measurements are likely to appear in the delay change and forwarding anomaly reports. And networks hosting at least ten public probes appear in the network disconnection reports. The best way to ensure that your network is monitored is to host an anchor (delay changes, forwarding anomalies) and at least ten probes (network disconnection).

For routing changes, we use BGP data to monitor the Internet routing infrastructure. If your AS is publicly advertising IP space, then it should be already monitored by our system.

What do the graphs mean?

Each graph represents a different aspect of the collected network data, ranging from routing changes to delay performance. Each graph is documented at the following links:

Also most of the graphs are interactive, click on them to get more details and obtain the API link to raw data.

Under the hood

Our monitoring tools are open source (https://github.com/InternetHealthReport) and described in the following articles:

Citation policy

If you publish material using data from the Internet Health Report, then help others to obtain the same data sets and replicate your experiments by citing the above papers.

Frequently Asked Questions (FAQ)

What is the time zone of displayed data?

All IHR graphs show UTC times. The IHR API also provides UTC times. However, some RIPE widgets might display local times (e.g. BGPlay).

Why is there no graph and no API results for certain networks?

Some of the IHR analysis modules analyze data from the RIPE Atlas platform. If a network hosts no Atlas probe or it doesn't appear in Atlas traceroutes, it is probably not appearing in our delay and disconnection results. A network should host at least ten probes to be included in our network disconnection analysis. Hosting an Atlas anchor is a good way to be part of our link delay and forwarding anomalies. Please see the RIPE website for applications to host a probe or an anchor.

Why is there no IXP in the AS dependency results?

AS dependency is computed from BGP data and IXPs are usually not visible in BGP paths. Thus, we currently cannot infer dependency to IXPs. This issue could be addressed by analyzing traceroutes instead of BGP's AS paths, which is a direction we are currently exploring.

What are the numbers assigned to IXPs?

IHR uses unique identifiers to index results for IXPs. These identifiers are derived from the ones found in CAIDA's IXPs dataset (https://www.caida.org/data/ixps/). To differentiate these identifiers with AS numbers, the API gives negative values to IXPs and positive values to ASes.
The list of all IXP names and identifiers is available at the following link: https://ihr.iijlab.net/ihr/api/networks/?number__lte=-1

How can I report bugs or suggest new features?

The source code of all IHR tools is hosted on GitHub. You can review and submit bug reports for the IHR website here. If you don't have a GitHub account contact us directly.

How can I contribute to IHR?

IHR is mainly built and maintained by researchers and students. We need help at all levels for the frontend, backend, documentation, and data collection. Feel free to take a look at IHR projects on GitHub or contact us. The IIJ Research Laboratory summer internship program is also a good way to visit us and work on IHR tools.

Acknowledgements

We are truly grateful to organizations and people that made IHR possible.

IIJ

Internet Initiative Japan (IIJ) is historically Japan's first ISP and the main sponsor of IHR. This includes financial support, hardware, and rack space.

IIJ Research Laboratory

IIJ Research Laboratory is the home of IHR. The IIJ Research Laboratory summer internship program has enabled invaluable students exchange for implementing and maintaining IHR projects.

Google Summer of Code

Google Summer of Code supports new contributors into the development of IHR.

RIPE NCC

IHR is built on top of large datasets and RIPE NCC is our main provider. Data from RIPE's Routing Information Service (RIS) and Atlas measurement platform is crucial for us. IHR has also been partly founded by the RIPE NCC Community Projects Fund.

Route Views

The University of Oregon Route Views Project is another key source of data for IHR, in particular for computing AS dependency.

Internet Society

Internet Society supported the development of an IHR tool to monitor the AS dependency of countries and increase IHR's computational power.

MANRS

Mutually Agreed Norms for Routing Security (MANRS) is a global initiative that supported the development of IHR tools to monitor route origin validation via the MANRS Fellowship Program.

Edgecast

Edgecast is a Content Delivery Network (CDN) that distributes IHR's website and API worldwide.

Global report

The global report is a summary of alarms detected across all networks monitored by IHR. This report is specifically designed to provide an overview of the overall conditions of the Internet at a particular point in time. Anomalous metrics are visible via the overview switches or by following links to the corresponding network reports. In order to limit the amount of displayed data, the general report shows only the most prominent alarms. Smaller alarms might be visible only on network reports.

Network report

Network reports provide all information collected by IHR about a selected network (AS or IXP). Network reports are available through the search bar at the top of the IHR webpages or by following links from the global report. Here are links to a few reports:

Country Report

A country report provides information collected by IHR about a certain country. Country reports are available through the search bar at the top of the IHR webpages. Here are links to a few reports:

AS dependency

What is AS dependency?

Networks connected to the Internet rely on other networks (a.k.a ASes) to transmit data. Consequently, the connectivity of a network depends on the connectivity of other networks. AS Hegemony is a metric to evaluate these interdependencies based on BGP data. See our research report for more details: https://www.iijlab.net/en/members/romain/pdf/romain_pam2018.pdf

What do the graphs mean?

Let's take a look at the above example which corresponds to results obtained on March 2nd 2020 for AS2497 (IIJ).

The top plot shows the AS dependencies of AS2497, i.e., the most common transit networks used to reach AS2497. The value (a.k.a AS Hegemony) ranges between 0 and 1. This is an estimate of the ratio of AS paths towards AS2497 that go through a certain AS transit. A value close to 1 means that the reachability of the monitored AS is strongly dependant on that transit AS. In the above example, the plot shows a few weak dependencies to Telia (AS1299), NTT America (AS2914), Level(3) (AS3356), and GTT (AS3257). On that day we estimate around 16% of the paths towards IIJ transit through Telia, 9% through NTT, and 2% through Level(3).

Stub ASes usually have much higher values, for example the values go up to 1 for ASes with a single upstream provider (meaning that all the paths cross the upstream provider). You can click on the graph to display dependency details at a particular timestamp and a graphical representation of BGP changes around that time (using BGPlay).

The bottom plot shows the number of networks that depend on AS2497. In this case there are more than 370 networks that use IIJ for transit. This is somehow related to IIJ's customer cone. The main difference is that networks using IIJ as a backup transit are not counted here whereas they are counted in its customer cone. You can click on the graph to obtain the complete list of dependent networks.

Known issues

Level shift for an entire day. Sometimes we cannot get as much data as we'd like. If the amount of data is very low, then graphs might show significant level shifts for the entire day.

References

Country's network dependency

A Country network dependency is computed in two different ways, emphasizing either the distribution of the country's population or the country ASes.

Population coverage

The population coverage combines two data sources, the estimated population for each AS and the paths from the Internet to these ASes. The total value represents the percentage of the population that is reached through the corresponding AS. This value is also dissected into two parts:

  • Direct is the percentage of the population directly connected to this AS
  • Indirect is the percentage of the population that is reached via this AS but that is not directly connected to it.

AS coverage

The AS coverage is the percentage of the country's ASes that are reached via this AS. Large values represent significant transit networks for the monitored country.

References

Route Origin Validation

For any route seen on BGP (RIS and Route Views) we report its RPKI and IRR status, as well as the status of the prefix and origin ASN in Regional Internet Registries Statistics files. In addition, we also compute the visibility of the prefix, that is the percentage of BGP speakers in RIS and Route Views that advertise the prefix, and the common transit networks found in AS paths.

Status

Following mostly the nomenclature given in RFC6811 the statuses for RPKI and IRR are:

  • NotFound
  • Invalid
  • Invalid,more-specific
  • Valid
The status Invalid,more-specific corresponds to routes that are invalid only because of the maximum prefix length attribute.

Using RIRs' delegated files the statues for the announced prefix and origin ASN are
  • assigned
  • available
  • reserved
  • allocated
  • ianapool
  • NotFound
Statuses other than assigned usually correspond to bogons.

Visibility corresponds to the percentage of RIS and Route Views peers that observe the route.

AS dependency

The main transit ASes for each prefix are identified using the AS Hegemony metric. The AS Hegemony value roughly corresponds to the percentage of AS paths that contain the transit AS (except for the API where the AS Hegemony is expressed as a value between zero and one).

Data freshness

This dataset is updated daily and reflects routing data has observed at midnight UTC.

Network delay

Using traceroute data collected with the RIPE Atlas measurement platform, we compute network delays between pairs of ASes, Atlas probes, and cities (IP addresses are geolocated with RIPE IPmap).

What does the graph mean?

By default graphs load network delay from the selected network to a few locations (Tokyo, London, Singapore, Ashburn, K-Root server (AS25152), Google (AS15169)), but you can load more results with the search fields above the graph. The graph shows the estimated median RTT from a network to these locations. To compute these values we do the following: We collect a set of RTT estimates by taking all RTT values in traceroute and computing differential RTTs (i.e. the difference between two RTT values found in a traceroute). Then we store the median value of these RTT estimates as a representative value. The above example depicts the median RTTs from an ISP in Singapore (AS24482) to pre-selected locations. As an example, the values for London represent the median values of all RTTs from IP addresses that map to AS24482 (that could be Atlas probes or router addresses) to IPs in the same traceroutes that map to London.

The example above shows two anomalies, the first one on March 16th represents a delay increase only to London, whereas the second one on March 17th impacted all plotted locations.

Known issues

Large ASes. Our methodology to collect and aggregate RTT values is relevant mainly for small ASes and cities. The median RTT from large ASes, e.g. Level(3), is misleading because these networks are spread out around the globe so they mostly have IP addresses far from the destination location. This technique works best with ASes concentrated on a geographical area.

Negative values. Because we are computing differential RTTs we may obtain negative values in some cases.

Delay and forwarding anomalies

Using traceroute data collected with the RIPE Atlas measurement platform, we detect, per link, delay changes and routing changes. The techniques we used are detailed in the following research paper: R. Fontugne, E. Aben, C. Pelsser, R. Bush, Pinpointing Delay and Forwarding Anomalies Using Large-Scale Traceroute Measurements. ACM IMC 2017, London, UK.

What does the link delay graph mean?


Let's take a look at the above example, which corresponds to results obtained on March 4th 2020 for Comcast (AS7922). The top plot (blue line) represents the overall delay changes we observe for AS7922's links. This value is normally close to 0, but features higher values when we observe significant delay changes for IP addresses in that AS. In this example we see that AS7922 exhibits increased delays around midnight on March 4th. By clicking on the peak at 1:30am we obtain the list of IP addresses that are detected by our system (bottom table) and the end-to-end delays for traceroute monitoring reported IP addresses. Here we see that end-to-end delays of traceroutes crossing the first reported links have increased by 60 ~ 90 ms. The red vertical dashed lines also show that packets are dropped at that time.

What does the forwarding anomaly graph mean?

For forwarding anomalies let's look at another example, AS174 Cogent on November 2017.

The bottom plot represents the forwarding anomalies observed for AS174. Intuitively, this graph represents the presence and absence of IP addresses for AS174 in the monitored traceroutes. If the number of observed IP addresses for AS174 is constant then the forwarding anomaly level is close to 0. If we observe more IP addresses than usual then the forwarding anomaly level takes higher positive values. On the other hand if we observe fewer IP addresses than usual, the forwarding anomaly level takes lower negative values. In this example, IP addresses from AS174 are appearing more than usual due to an internal routing anomaly. You can click on the graphs to obtain the list of reported IP addresses, we used to have a visualization of corresponding traceroutes (TraceMON) but this currently not working.

Known issues

Traceroute limitations. Our analysis of delay and forwarding anomalies relies solely on traceroute results. That implies a few limitations, for example, unexpected results for MPLS tunnels that do not implement RFC4950.

References

Network disconnections

To detect significant network disconnections we monitor the status of Atlas probes. We essentially seek synchronized disconnections of probes that are in the same geographical or topological area. See our research report for more details: A. Shah, R. Fontugne, E. Aben, C. Pelsser, R. Bush, Disco: Fast, Good, and Cheap Outage Detection. TMA 2017

What does the graph mean?


The above example shows the disconnection of an Iranian network (AS16322) due to an update in an upstream provider on March 3rd 2020. The world map shows the location of disconnected Atlas probes and the below graphs show ping results from these probes. These ping measurements provide us with an internal view of the outage. Here we found that pings toward Google DNS and Atlas controller failed from around 00:45 until 01:35. Pings to the K-root server are, however, carried out as usual. This means that this network could still reach a K-root instance during the outage (probably the one located in Teheran).

The table reports a deviation score, 11 in this case, and the number of disconnected Atlas probes. Larger deviation values represent stronger evidence of the network disconnection. To avoid too many false positives, only higher values are reported (this threshold is different for the global and network reports). We also only monitor networks and countries that have at least five Atlas probes.

Known issues

Atlas infrastructure. Sudden Atlas controller reboots or outages near Atlas controllers may result in large-scale Atlas probes disconnection thus appear as a large outage for our detector. This usually results in failed pings to the Atlas controllers whereas other pings reach their final destination as usual.

References

REST API

All results displayed on the Internet Health Report website are available via the IHR REST API. For quick access to plotted data just click on the graph and see the API tab below the graph.

Format

Two formats are available, HTML and JSON. The HTML format allows developers to easily play with the API. The JSON format provides a programmatic access to our reports. The results are formatted in HTML if you access the API with your web browser, JSON is used otherwise.

Endpoints

The API is accessible at http://ihr.iijlab.net/ihr/api/, there are multiple endpoints available, which are documented in the API documentation.

Filtering

You can filter your search by adding parameters in the URL. For example:

You can also filter results using inequalities. Add the suffix '_gte' and '_lte' for 'greater than or equal' and 'less than or equal'. For example, https://ihr.iijlab.net/ihr/api/network_delay/?asn=174&timebin__gte=2017-06-01T00:00&timebin__lte=2017-07-01T00:00 returns delay change results for AS174 on June 2017.

Scripting

This is a python example to fetch the latest delay change results for AS174 on June 2017:

import sysimport requests
url = 'https://ihr.iijlab.net/ihr/api/network_delay/'
params={
    'asn': 174
    'timebin__gte': '2017-06-01T00:00'
    'timebin__lte': '2017-07-01T00:00'} 
resp = requests.get(url,params) 
if (resp.ok): 
    data = resp.json()
    print(data) 
else: 
    resp.raise_for_status()

Python Library

To ease the access to the IHR API we also provide a python library named abondance.

Installation

Installing the library requires only the standard python package manager, pip. From a terminal type:

pip install abondance

Example:

Here is a simple example to fetch and print AS dependency results for AS2501 on September 15th, 2018.

from ihr.hegemony import Hegemony
hege = Hegemony(originasns=[2501], start='2018-09-15 00:00', end='2018-09-15 23:59')
for r in hege.get_results():
    print(r)
For more examples visit https://pypi.org/project/abondance/

Daily dumps

For users that need faster access to more IHR data, we also provide daily dumps of IHR database. Please refer to https://ihr-archive.iijlab.net

Data policy

The goal of IHR is to advance network research and empower the public with useful information about the characteristics of networks that constitute the Internet. IHR data is provided 'as is,' with all faults, defects, bugs, and errors. IHR does not make any warranty as to the accuracy or completeness of the data.

Internet Health Report is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Permissions beyond the scope of this license may be available at admin@ihr.live.