# Monitoring AI tools

AI Tools dashboards offer detailed visibility into how AI tools are adopted, used, and perceived across your organization. By combining operational data with sentiment and employee feedback, you can optimize and scale AI adoption.

## Analyzing global AI adoption from the Overview dashboard

{% hint style="warning" %}
The system estimates **AI usage** on desktop-type applications using the `focus_time` field from the `execution.events` . If the `focus_time` opt-in field is not [enabled](https://docs.nexthink.com/platform/user-guide/getting-started-with-ai-tools#before-you-begin), Nexthink cannot monitor user interactions with AI tools on desktop applications.
{% endhint %}

The **AI Tools Overview** dashboard aggregates insight across all AI tools, including both commercial and custom-built tools.

The system automatically monitors preconfigured AI tools out of the box—such as ChatGPT—through traffic pattern recognition, endpoint activity, and employee feedback.

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2FboRgAFnCmkVvmH6QKfx1%2Fimage.png?alt=media&#x26;token=4094fdbc-7856-4616-af0c-f710169022f4" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
The system displays the ✦ sparkles icon to indicate AI-generated content or insights. AI is evolving rapidly and delivering great insights, but it can still make mistakes. Nexthink recommends validating your results to ensure accuracy and support informed decision-making.

Refer to the [Nexthink Insights - AI Model Card](https://docs.nexthink.com/legal/global-ai-hub/nexthink-insights-ai-model-card) documentation for more information.
{% endhint %}

To analyze global AI adoption:

1. Navigate to **AI Tools** > **AI Tools Overview** from the main navigation.
2. Explore the top section of the **AI Tools Overview** dashboard with **AI adoption**, engagement metrics, and **Industry benchmarks**:
   * **Weekly** **AI-engaged time per employee**—with bar chart visualization.
   * Weekly active AI user percentages and average number of tools used per employee.
   * Percentage of **inactive users** who have not used AI tools in the last year.
   * The number of AI tools that were used per employee in the past 30 days.
   * AI-generated **Insights** with recommendations to address usage or enablement gaps.

<details>

<summary>How does the system estimate <strong>AI-engaged time</strong>?</summary>

The system estimates the AI-engaged time per employee using **15-minute intervals** or buckets.

* Any use or interaction with an AI tool within a 15-minute bucket counts as 15 minutes of AI-engaged time.
* Multiple interactions with AI in the same interval still count as one 15-minute bucket.

Therefore, brief or occasional use of AI tools could add up to a significant weekly total—see the **example below**.

{% hint style="info" %}
The **15-minute intervals** are designed to capture employees' adoption and integration of AI into their daily work, rather than a precise minute-by-minute usage of AI tools.
{% endhint %}

<div align="left"><figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2FGzazaPb2aNder3KsF54j%2Fimage.png?alt=media&#x26;token=d5d9858e-82c4-43c9-afd6-e81c9338b57d" alt="" width="563"><figcaption></figcaption></figure></div>

</details>

{% hint style="info" %}
For AI tools preconfigured by Nexthink out of the box, the system highlights adoption metrics that are below or above the **Industry benchmark**, easily identified by red and green color codes.

Hover over each benchmark indicator for further information.
{% endhint %}

3. Review the **Employee experience with AI tools,** collected from [built-in Nexthink campaigns](#built-in-campaigns-for-monitoring-employee-perception-of-ai-tools).
   * View the average number of hours employees believe they save each week by using AI tools. Compare the perceived saved time to **Industry benchmarks**.
   * Identify successful **Use cases** based on employee comments on AI tool usage in their daily work, for possible scalability. These use cases are AI-generated by Nexthink based on campaign responses.
   * Execute **Recommended actions** to address employee-reported challenges when using AI tools. These actions are AI-generated by Nexthink based on campaign responses.
4. Leverage **AI usage details** tables and widgets to sort AI adoption and engagement metrics by AI tool, or break usage down by **Department** or [configured custom filters](https://docs.nexthink.com/platform/user-guide/configuring-ai-tools#configuring-custom-filters-for-ai-tools-dashboards) that define your organizational hierarchy groups.
   * Visually break down **AI usage patterns** into behavior quadrants to identify where enablement or optimization is most needed:
     * **Champions**: high usage time and frequency.
     * **Deep divers**: high usage time, lower frequency.
     * **Regulars**: steady, moderate usage.
     * **Explorers**: low usage time and frequency.
     * The size of the circles in the quadrant chart represents the number of employees using Al tools in each department.
   * Detect adoption patterns across AI tools and organizational units using a heatmap for multidimensional data visualization.
     * Select the adoption metric to **Display**, such as **Weekly active users**, **Inactive users**, **Weekly engagement time** with AI tool&#x73;**,** and **Perceived weekly time saved** from the employee perspective.
     * View the adoption metrics by **Departments** or by configured custom filters that define your organizational hierarchy groups.
     * Spot anomalies, such as tools with low activity in high-priority divisions, and simplify decision-making through a colored representation.
5. Gain insights into user frequency for **Tools in use** **(Last 7 days)** across your organization:
   * Number of **Active users** per AI too&#x6C;**.**
   * **AI-engaged time per employee** with each tool (in hours). Refer to [#how-does-the-system-estimate-ai-engaged-time](#how-does-the-system-estimate-ai-engaged-time "mention").
   * **Industry benchmark** of AI-engaged time, per employee, with each tool (in hours).&#x20;
   * **Total AI-engaged time** represents the time all employees spent interacting with the tool.
6. Optionally, apply filters to the entire dashboard using the **Department** filter at the top of the page.
   * If needed, you may [configure custom filters](https://docs.nexthink.com/platform/user-guide/configuring-ai-tools#configuring-custom-filters-for-ai-tools-dashboards) for AI Tools dashboards.

After detecting global AI adoption patterns and gaps, you should further investigate using tool-specific dashboards.

***

## Examining the adoption of a particular AI tool from a tool-specific dashboard

{% hint style="warning" %}
The system estimates **AI usage** on desktop-type applications using the `focus_time` field from the `execution.events` . If the `focus_time` opt-in field is not [enabled](https://docs.nexthink.com/platform/user-guide/getting-started-with-ai-tools#before-you-begin), Nexthink cannot monitor user interactions with AI tools on desktop applications.
{% endhint %}

Every AI tool features a dedicated dashboard that provides detailed insights into its usage and adoption metrics. This allows you to adjust license numbers, initiate enablement campaigns/training, or identify low usage areas.

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2FHy2jEBVupAM3dj9w38eO%2Fimage.png?alt=media&#x26;token=9e26772a-eea5-4033-89f1-3a031a857967" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
The system displays the ✦ sparkles icon to indicate AI-generated content or insights. AI is evolving rapidly and delivering great insights, but it can still make mistakes. Nexthink recommends validating your results to ensure accuracy and support informed decision-making.

Refer to the [Nexthink Insights - AI Model Card](https://docs.nexthink.com/legal/global-ai-hub/nexthink-insights-ai-model-card) documentation for more information.
{% endhint %}

To analyze the adoption of a specific AI tool:

1. From the navigation menu, select the specific AI tool of interest listed under **AI Tools**.
2. Explore the tool-specific dashboard, designed to resemble the layout of the [AI Tools Overview](#analyzing-global-ai-adoption-from-the-overview-dashboard) dashboard, combining tool-specific **AI adoption**, engagement metrics, and **Industry benchmarks** with:
   * **AI usage details** tables and widgets with tool-specific adoption breakdowns by **Department** or [configured custom filters](https://docs.nexthink.com/platform/user-guide/configuring-ai-tools#configuring-custom-filters-for-ai-tools-dashboards) that define your organizational hierarchy groups.
   * **Weekly active users (WAU):** The percentage of users who most frequently engage with the specific AI tool.
   * Percentage of **Churned users** who have not used the specific AI tool in the last 30 days, but used it at least once in the last year.
   * Tool-specific available **licenses**.
   * AI-generated **Insights** with recommendations to address tool-specific usage and enablement gaps.

{% hint style="info" %}
For AI tools preconfigured by Nexthink out of the box, the system highlights adoption metrics that are below or above the **Industry benchmark**, easily identified by red and green color codes.

Hover over each benchmark indicator for further information.
{% endhint %}

3. Review the **Employee experience with the AI tool,** collected from [built-in Nexthink campaigns](#built-in-campaigns-for-monitoring-employee-perception-of-ai-tools).
   * View the average number of hours employees believe they save each week by using the AI tool. Compare the perceived saved time to **Industry benchmarks**.
   * Identify successful **Use cases** based on employee comments on AI tool usage in their daily work, for possible scalability. These use cases are AI-generated by Nexthink based on campaign responses.
   * Execute **Recommended actions** to address employee-reported challenges when using the AI tool. These actions are AI-generated by Nexthink based on campaign responses.
4. Leverage tool-specific advanced visualizations and benchmarking of **AI-engaged time distribution** and user-tool **interactions**, helping to better assess employee usage and feedback.
   * The **AI-engaged time Distribution** chart illustrates the percentage of employees using the AI tool within various time intervals (0–2 hours, 2–4 hours) up to—but not including—40 hours per week. Refer to [#how-does-the-system-estimate-ai-engaged-time](#how-does-the-system-estimate-ai-engaged-time "mention").
   * The **AI interactions** chart displays the time-series trend and the progression in the volume of interactions.
   * Visually break down tool-specific **AI usage patterns by Department** or configured custom filters, into behavior quadrants to identify where enablement or optimization is most needed.
     * **Champions**: high usage time and frequency.
     * **Deep divers**: high usage time, lower frequency.
     * **Regulars**: steady, moderate usage.
     * **Explorers**: low usage time and frequency.
     * The size of the circles in the quadrant chart represents the number of employees using the specific Al tool in each department.
5. Optionally, apply filters to the entire dashboard using the **Department** or [Application type](https://docs.nexthink.com/platform/user-guide/configuring-ai-tools#configuring-custom-ai-tools-for-nexthink-monitoring)—**Web** or **Desktop** version of the AI tool—at the top of the page.
   * If needed, you may [configure custom filters](https://docs.nexthink.com/platform/user-guide/configuring-ai-tools#configuring-custom-filters-for-ai-tools-dashboards) for AI Tools dashboards.

<details>

<summary>Applying filters specific to the <strong>Microsoft Copilot</strong> dashboard</summary>

From the **Microsoft Copilot** dashboard, filter by **Copilot type**:

* The **Microsoft 365 Copilot** filter displays tool-specific data for licensed Copilot usage.
* The **Copilot chat** filter displays tool-specific data for unlicensed Copilot usage.

</details>

<details>

<summary>Analyzing widgets specific to the <strong>Microsoft Copilot</strong> dashboard</summary>

Use specific widgets available in the **Microsoft Copilot** dashboard:

* The **Applications in use (Last 7 days)** widget shows usage breakdown of MS Copilot across applications, such as SharePoint, Excel, PowerPoint and more.
  * Check if your organization is **Leading** or **Lagging** in terms of **AI-engaged time per employee** on a weekly basis compared to industry benchmarks.
  * Keep track of the week-over-week (**WoW**) AI interaction trend in Copilot usage for specific applications.

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-8db39ba829057e52f15e691b7ac9aac5354023e9%2Fimage.png?alt=media" alt=""><figcaption></figcaption></figure>

</details>

<details>

<summary>Detecting AI engagement patterns specific to the <strong>Microsoft Copilot</strong> dashboard</summary>

Detect AI engagement patterns of Microsoft Copilot across organizational units and applications by using a heatmap that displays weekly AI engagement time by department and application.

Spot anomalies, such as tools with low activity in high-priority divisions, and simplify decision-making through a colored representation.

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-818a1313b831e2fe126c18d26f41057bf542c4e6%2Fimage.png?alt=media" alt=""><figcaption></figcaption></figure>

</details>

### Built-in campaigns for monitoring employee perception of AI tools <a href="#campaigns-for-monitoring-perception" id="campaigns-for-monitoring-perception"></a>

{% hint style="warning" %}
Ensure all employee devices have Collector version `25.5.1` or later. This guarantees the correct deployment of built-in Nexthink campaigns.
{% endhint %}

To display the **Employee experience with AI tools** in all AI Tool dashboards, Nexthink uses built-in campaigns that you should [enable and activate](https://docs.nexthink.com/platform/user-guide/configuring-ai-tools#configuring-ai-tools-for-nexthink-monitoring) from the specific AI tool configuration. Once the campaign is activated, allow up to seven days for the data to appear in the AI tool dashboard.

Powered by AI, Nexthink generates **Recommended actions** and **Use cases** based on employee feedback. In addition, the system compares the perceived saved time to **Industry benchmarks**.

Open the **Employee experience campaign responses** right-side panel to:

* Review each employee comment on Recommended actions individually, including a summary at the top under **Insights.**
* Review each employee comment on Use cases individually, including a summary at the top under **Insights.**

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2FOq6Z8z4yp6e6iARF13hk%2Fimage.png?alt=media&#x26;token=a25b24b3-daaf-4ee5-a764-2549f4621802" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
The system displays the ✦ sparkles icon to indicate AI-generated content or insights. AI is evolving rapidly and delivering great insights, but it can still make mistakes. Nexthink recommends validating your results to ensure accuracy and support informed decision-making.

Refer to the [Nexthink Insights - AI Model Card](https://docs.nexthink.com/legal/global-ai-hub/nexthink-insights-ai-model-card) documentation for more information.
{% endhint %}

The system sends AI tools campaigns with uneditable content adhering to the following standard:

<details>

<summary>Questions included in each campaign</summary>

Each campaign includes these **three questions**:

1. *How much time has* `[AI tool]` *saved you over the past 7 days?*
   * Options: Time ranges (*None, Less than 1h, 1-5h, 6-10h, 11-15h, 15-20h, More than 20h*)
2. *How could* `[AI tool]` *be more useful to you, and what could we do to help?* *(*&#x46;ree text input).
3. *What is the last thing you did, or tried to do, with* `[AI tool]`*? (*&#x46;ree text input).

</details>

<details>

<summary>Criteria for sending campaigns</summary>

Campaigns are sent under the following conditions:

* Only employees actively using at least one monitored AI tool receive campaigns.
* Each employee receives a campaign at most once every 90 days, and only for one of the tools they use.
* A maximum of 5% of the employees receive a campaign on any given day.

If needed, you can [modify campaign settings for AI tools](https://docs.nexthink.com/platform/user-guide/managing-ai-tools#configuring-campaign-setups-for-ai-tools) to exclude specific users from campaigns or disable the campaign entirely.

</details>

***

## Balancing AI adoption

Report adoption results of AI tool solutions to stakeholders and leadership:

* Track employee-AI interaction trends to optimize the allocation of licenses/resources.
* Measure employee response shifts after conducting suggestions from AI Tools **Insights** and **Recommended actions**.
  * Validate the effectiveness of targeted communication or training for specific AI tools.
  * Track whether walkthroughs or [Adopt](https://docs.nexthink.com/platform/user-guide/adopt) guides drive measurable improvements.

Ultimately, you should observe tangible value increments as a result of AI tool adoption.

{% hint style="warning" %}
To maintain objectivity in reporting, follow Nexthink dashboards and insights to identify correlations and isolate variables. This way, you avoid over-attributing outcomes to AI tool adoption efforts without sufficient validation.

In addition, support observations with evidence-based frameworks such as statistical correlation measures, e.g., Pearson’s R or controlled comparisons (A/B testing).
{% endhint %}

***

RELATED TASKS

* [configuring-ai-tools](https://docs.nexthink.com/platform/user-guide/ai-tools/configuring-ai-tools "mention")
* [managing-ai-tools](https://docs.nexthink.com/platform/user-guide/ai-tools/managing-ai-tools "mention")