Skip to main content
Skip table of contents

Application Connectivity troubleshooting

Problem

Reporting and acting on network-related issues requires accurate data collection, visualization and interpretation. Without reliable network performance indicators, fixing connection issues becomes a guessing game. Ultimately, leading to poor employee experience and resource wasting.

Solution

The Application Connectivity troubleshooting framework is a set of investigation principles and pre-defined NQL queries to help you:

  • Identify the root causes behind network-related issues with absolute certainty.

  • Effectively troubleshoot network-related issues with targeted solutions (hold associated parties accountable).

  • Control device data privacy and ensure compliance within your organization.

To achieve this, the Application Connectivity framework relies on connections data with connection events, internet destinations, and data privacy compliances at an application-device level.

The NQL queries on this page enable accurate reporting of connection statistics. Similarly, these queries are supported by different query-based features available in the Nexthink web interface.

You can also use Network view for connections data visualization, filtering and drill-downs (transport protocols, devices, binaries, destinations, etc.).

Prerequisites

  • Connections events are only available for devices with Collectors that report 'Infinity only'.

  • Minimum Collector version of 2023.10.

Connections data

The connections data used by Network view and the Application Connectivity NQL queries on this page include:

  • Connection events

  • Connection events decoration

  • Connection events aggregation

Jump to the Network troubleshooting with Connections Data section to learn about the use of Application Connectivity queries.

Connection events

A connection event represents an outgoing TCP connection (transmission control protocol established by a device) or an outgoing UDP package (user datagram protocol). Each connection event provides the following information:

  • Start time, end time and duration of the event’s bucket

  • The source of the connection event

  • The destination of the event

  • The transport protocol and IP version

  • Metrics about the connection

Connection events are sampled events, meaning Nexthink reports connection events in buckets of 15 minutes and 1 day.

The namespace connection of the NQL data model contain three main tables:

  • The connection.events table contains events for outgoing TCP connections and UPD packages. This table only contains metrics standard to TCP and UDP.

  • The connection.tcp_events table contains events for outgoing TCP connections.

  • The connection.udp_events table contains events for outgoing UDP packages.

Refer to the NQL data model documentation for more information

Connection events decoration

Nexthink decorates connection events data with connection source, connection destination and connection metrics.

Connection Source

Connection events hold information about the source of a connection:

  • The device that establishes the connection.

  • The binary that uses the connection.

  • The user of the process that runs the binary.

  • Optionally, the desktop application configured for this binary.

Connection Destination

The IP and the network port define the destination of a connection. Additionally, Nexthink decorates connection events data with a destination type, the 'subnet address' and optional information:

  • Domain name of the destination.

  • Owner of the destination.

  • Country and location of the destination (GeoIP information).

  • Datacenter region name provided by the owner of the destination.

The destination subnet address equals the IP address with the last 8 bits set to zero.

Nexthink uses GeoIP data and published IP address ranges to enrich the destination information of connections. See the table below.

Condition

Connection Type

Destination Information

Private IP address

intranet

No additional destination information available

Public IP address

internet

  • destination owner: name provided by the owner of the Autonomous System the IP address belongs to

  • destination country: based on GeoIP information

Public IP address belonging to the range of IP addresses published by the destination owner.

datacenter

  • destination owner: name for the owner of the IP range

  • destination country: based on GeoIP information

  • data center region: name for the data center provided by the owner

Information about the destination owner, country, and data center region is not always available or partially available only. The corresponding fields are NULL in this case.

Connections Metrics

Connection events provide the following metrics:

  • The connection round-trip-time (RTT). That is, the round-trip-time of the first two packages during the TCP connection establishment handshake. This metric is only available for TCP events with at least one established connection.

  • Incoming and outgoing traffic in bytes. Data received (TCP only) and sent (TCP and UDP) by the application during the event.

  • Number of connections per status in the event.

Cardinality per connection status

  • Successful or alive connections are the number of established connections in a previous time bucket and continue into the current time bucket. (TCP and UDP)

  • Failed connections (TCP only) can be one of the following:

    • Connections failed with no host: The number of connections that failed due to the device not reaching the destination host.

    • Connections failed with no service: The number of connections that failed due to the device not reaching the service on the destination host.

    • Rejected connections: The number of outgoing connections that have been rejected on the device of the user.

  • The number of connections that Nexthink aggregates into one event is the total number of failed and successful connections.

Connection events aggregation

Nexthink aggregates connections into buckets of 15 minutes and 1 day.

15-min aggregation

Nexthink aggregates into one event all connections happening within the same 15-minute bucket that share the following characteristics:

  • Same device

  • Same binary

  • Same user running the binary

  • Same IP address and network port

  • Same transport protocol (TCP or UDP)

When aggregating connections into a 15-minute bucket, Nexthink keeps the following:

  • The average of the connection round-trip times for established TCP connections.

  • The total sum of incoming and outgoing traffic of all connections in the bucket.

  • The number of connections per status.

1-day subnet aggregation

When Nexthink aggregates 15-minute buckets into 1-day buckets, the system sets the IP address to NULL and combines all 15-minute events into one 1-d event that share the following characteristics:

  • Same device

  • Same binary

  • Same user running the binary

  • Same subnet mask and network port

  • Same transport protocol (TCP or UDP)

  • Same destination fields

When aggregating connections into a 1-day bucket, Nexthink keeps the following:

  • The weighted average of the connection round-trip times for established TCP connections.

  • The total sum of incoming and outgoing traffic of all connections in the bucket.

  • The number of connections per status.

Network troubleshooting with Connections data

Use connections data to troubleshoot network-related issues. Follow these main steps:

  1. Identify the relevant population (devices, apps, destinations) affected by network issues.

  2. Investigate TCP connections.

  3. Investigate UPD connections.

Find in this section NQL queries for each of the main troubleshooting steps.

You can apply the same troubleshooting principles to Network view for connections data visualization, filtering and drill-downs (transport protocols, devices, binaries, destinations, etc.).

Identifying the relevant population

Focus on three dimensions of the connections data:

  • Device Dimension: Which and how many are the impacted devices?

    • Determine the impacted devices sharing the same characteristics and location.

  • Application Dimension: Which and how many are the impacted desktop applications?

  • Destination Dimension: Which are the impacted destinations? Where are they located?

Device Dimension

Connections events are linked to the device object that created the network connection. This allows you to investigate the connections data of a single device (by devices.name) or a group of devices, for example by devices.entity or other custom organizational unit classifications.

Refer to the Product configuration documentation for more information.

To group devices by location, use the location context of the connection event, for example:

CODE
connection.tcp_events during past 7d
| where binary.name == "*outlook*"
| summarize Avg_RTT = establishment_time.avg() by context.location.country

The location context is where the device is at the time of the event. It requires an activated geolocation feature and works best when the collector traffic is routed to the Internet directly and not through a VPN.

Alternatively, you can use the organizational context, for example:

CODE
connection.tcp_events during past 7d
| where context.organization.entity == "XYZ"
| summarize Total_Connections = number_of_connections.sum() by application.name

Application Dimension

Connection events are linked to the binary object that initiates the connection. Additionally, the connection event is linked to a desktop application if the binary is part of the application definition.

Destination Dimension

The destination is a structured field of the connection event. For example:

CODE
connection.tcp_events during past 7d
| where destination.port == 135
| where number_of_failed_connections > 0

Note that It is impossible to summarize by IP address because the cardinality of IP addresses is too high.

Refer to the Destination documentation for more information.

Investigating TCP Connections

The two main metrics to gain visibility on the quality of TCP connections are:

  • The connections round-trip-time (RTT): The connections RTT is available for all TCP events with established connections and can be accessed through tcp_events.establishment_time.avg. Connections RTT is a good indicator for slow connections.

  • The failed connections ratio: The number of failed connections over the number of new connections (established and failed):
    Failed_Connections_Ratio = (number_of_failed_connections.sum()) / ((number_of_established_connections.sum()) + (number_of_failed_connections.sum()))

  • Often, failed connections ratio and its value fluctuation are better indicators of service availability than just monitoring failed connections.

Example: Investigation of VPN connectivity issues

Find an example below of a live dashboard to investigate VPN connectivity issues. Notice that the application dimension is fixed to the VPN binaries.

live dashboard to investigate VPN connectivity issues

Find below the NQL queries from the example of investigating VPN connectivity issues:

CODE
connection.tcp_events during past 24h
| where binary.name in ["VPN_binary_Windows", "VPN_binary_macOS"]
| where destination.domain == "VPN_edge_domain_name"
| where number_of_established_connections > 0
| summarize Devices__ = device.name.count(), Avg_RTT = establishment_time.avg(), Failed_Connections_Ratio_in_percent = (number_of_failed_connections.sum()) / ((number_of_established_connections.sum()) + (number_of_failed_connections.sum())) * 100 by context.location.country, destination.country, destination.datacenter_region
| sort Devices__ desc
CODE
connection.tcp_events during past 720min
| where binary.name in ["VPN_binary_Windows", "VPN_binary_macOS"]
| where destination.domain == "VPN_edge_domain_name"
| where number_of_established_connections > 0
| summarize average_RTT = establishment_time.avg() by 15min

Investigating UDP Connections

Because of the connectionless nature of UDP, Investigating UDP network traffic is limited compared to TCP network traffic. Your main tool is to look for changes and differences in the amount of outgoing UDP traffic, for example, comparing the average traffic per device for one application.

You can apply the same troubleshooting approach to Network view for connections data visualization, filtering and drill-downs (including transport protocols).

Application Connectivity in Nexthink Infinity

Find below the related documentation of some of the Nexthink features compatible with Application Connectivity’s connections data and queries:

You can use the Application Connectivity queries included on this page for all NQL-based features in the Nexthink web interface.

Overseeing data privacy

Refer to the Configuring Collector level anonymization documentation to anonymize, filter and control connections data privacy using NQL commands and sample queries.

In addition, find below details of how the Collector manages aspects related to data privacy:

Domain name reporting

The Collector retrieves the connection domain names from the TLS Server Name Indication (SNI) extension and also from monitoring DNS requests on the device (Windows only). The Collector does not report the domain name if it cannot retrieve the domain name from SNI or the application does not use the OS for the DNS look-up.

Some applications like Google Chrome web browser encrypt the SNI. This is called TLS Encrypted ClientHello (ECH). The Collector cannot report the domain name for connections making use of ECH. Users can disable ECH in the application to ensure the reporting of the domain name.

The Collector reports “multiple domain names" instead of a real domain name, in case of multiple connection events for the same binary and user, on the same device, at the same time, and to the same destination (same IP address and port) but different domain names.

Domain name shortening

The Collector shortens long and randomized domain names by evaluating various mathematical and heuristic properties.

Maximum number of connections per binary or user

The Collector reports up to 512 connections per binary and user within 5 minutes. If there are more than 512 connections for one binary and user, the collector only reports one event for all these connections for this time interval.

Connections through a VPN

VPN products implement various solutions to intercept and route connections. The Collector reports different and sometimes incomplete data depending on the VPN product, operating system of the device and configuration of the VPN solution.


RELATED TOPICS

RELATED TRAINING

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.