Monitoring AI tools

AI Tools dashboards offer detailed visibility into how AI tools are adopted, used, and perceived across your organization. By combining operational data with sentiment and employee feedback, you can optimize and scale AI adoption.

Analyzing global AI adoption from the Overview dashboard

The system estimates AI usage on desktop-type applications using the focus_time field from the execution.events . If the focus_time opt-in field is not enabled, Nexthink cannot monitor user interactions with AI tools on desktop applications.

The AI Tools Overview dashboard aggregates insight across all AI tools, including both commercial and custom-built tools.

The system automatically monitors preconfigured AI tools out of the box—such as ChatGPT—through traffic pattern recognition, endpoint activity, and employee feedback.

The system displays the ✦ sparkles icon to indicate AI-generated content or insights. AI is evolving rapidly and delivering great insights, but it can still make mistakes. Nexthink recommends validating your results to ensure accuracy and support informed decision-making.

Refer to the Nexthink Insights - AI Model Card documentation for more information.

To analyze global AI adoption:

Navigate to AI Tools > AI Tools Overview from the main navigation.
Explore the top section of the AI Tools Overview dashboard with AI adoption, engagement metrics and benchmarking:
- Weekly AI-engaged time per employee—with bar chart visualization.
- Weekly active AI user percentages and average number of tools used per employee.
- Percentage of inactive users who have not used AI tools in the last year.
- The number of AI tools that were used per employee in the past 30 days.
- AI-generated Insights with recommendations to address usage or enablement gaps.

For AI tools preconfigured by Nexthink out of the box, the system highlights adoption metrics that are below or above the industry benchmark, easily identified by red and green color codes.

Review the Employee experience with AI tools, collected from built-in Nexthink campaigns.
- View the average number of hours employees believe they save each week by using AI tools.
- Identify successful Use cases based on employee comments on AI tool usage in their daily work, for possible scalability. These use cases are AI-generated by Nexthink based on campaign responses.
- Execute Recommended actions to address employee-reported challenges when using AI tools. These actions are AI-generated by Nexthink based on campaign responses.

Benchmarking features for Employee experience with AI tools may not be available immediately for launch on October 30th, 2025—to be introduced at a later date.

Leverage AI usage details tables and widgets to sort AI adoption and engagement metrics by AI tool, or break usage down by Department or configured custom filters that define your organizational hierarchy groups.
- Visually break down AI usage patterns into behavior quadrants to identify where enablement or optimization is most needed:
  - Champions: high usage time and frequency.
  - Deep divers: high usage time, lower frequency.
  - Regulars: steady, moderate usage.
  - Explorers: low usage time and frequency.
  - The size of the circles in the quadrant chart represents the number of employees using Al tools in each department.
Detect adoption patterns across AI tools and organizational units using a heatmap for multidimensional data visualization.
- Select the adoption metric to Display, such as Weekly active users, Inactive users, Weekly engagement time with AI tools, and Perceived weekly time saved from the employee perspective.
- View the adoption metrics by Departments or by configured custom filters that define your organizational hierarchy groups.
- Spot anomalies, such as tools with low activity in high-priority divisions, and simplify decision-making through a colored representation.
Gain insights into user frequency for Tools in use across your organization:
- Weekly AI-engaged time per employee with each tool (in hours).
- Number Weekly active users per AI tool.
- Churned users are those who were active 30 days ago but have shown no activity since.
Optionally, apply filters to the entire dashboard using the Department filter at the top of the page.
- If needed, you may configure custom filters for AI Tools dashboards.

After detecting global AI adoption patterns and gaps, you should further investigate using tool-specific dashboards.

Examining the adoption of a particular AI tool from a tool-specific dashboard

Every AI tool features a dedicated dashboard that provides detailed insights into its usage and adoption metrics. This allows you to adjust license numbers, initiate enablement campaigns/training, or identify low usage areas.

Refer to the Nexthink Insights - AI Model Card documentation for more information.

To analyze the adoption of a specific AI tool:

From the navigation menu, select the specific AI tool of interest listed under AI Tools.
Explore the tool-specific dashboard, designed to resemble the layout of the AI Tools Overview dashboard, combining tool-specific AI adoption, engagement metrics, and benchmarking with:
- AI usage details tables and widgets with tool-specific adoption breakdowns by Department or configured custom filters that define your organizational hierarchy groups.
- Weekly active users (WAU): The percentage of users who most frequently engage with the specific AI tool.
- Percentage of Churned users who have not used the specific AI tool in the last 30 days, but used it at least once in the last year.
- Tool-specific available licenses.
- AI-generated Insights with recommendations to address tool-specific usage and enablement gaps.

For AI tools preconfigured by Nexthink out of the box, the system highlights adoption metrics that are below or above the industry benchmark, easily identified by red and green color codes.

Review the Employee experience with the AI tool, collected from built-in Nexthink campaigns.
- View the average number of hours employees believe they save each week by using the AI tool.
- Identify successful Use cases based on employee comments on AI tool usage in their daily work, for possible scalability. These use cases are AI-generated by Nexthink based on campaign responses.
- Execute Recommended actions to address employee-reported challenges when using the AI tool. These actions are AI-generated by Nexthink based on campaign responses.

Benchmarking features for Employee experience with AI tools may not be available immediately for launch on October 30th, 2025—to be introduced at a later date.

Leverage tool-specific advanced visualizations and benchmarking of AI-engaged time distribution and user-tool interactions, helping to better assess employee usage and feedback.
- The AI-engaged time Distribution chart illustrates the percentage of employees using the AI tool within various time intervals (0–2 hours, 2–4 hours) up to—but not including—40 hours per week.
- The AI interactions chart displays the time-series trend and the progression in the volume of interactions.
- Visually break down tool-specific AI usage patterns by Department or configured custom filters, into behavior quadrants to identify where enablement or optimization is most needed.
  - Champions: high usage time and frequency.
  - Deep divers: high usage time, lower frequency.
  - Regulars: steady, moderate usage.
  - Explorers: low usage time and frequency.
  - The size of the circles in the quadrant chart represents the number of employees using the specific Al tool in each department.
Optionally, apply filters to the entire dashboard using the Department or Application type—Web or Desktop version of the AI tool—at the top of the page.
- If needed, you may configure custom filters for AI Tools dashboards.

Applying filters specific to the Microsoft Copilot dashboard

From the Microsoft Copilot dashboard, filter by Copilot type:

The Microsoft 365 Copilot filter displays tool-specific data for licensed Copilot usage.
The Copilot chat filter displays tool-specific data for unlicensed Copilot usage.

Analyzing widgets specific to the Microsoft Copilot dashboard

Use specific widgets available in the Microsoft Copilot dashboard:

The Applications in use widget shows usage breakdown of MS Copilot across applications, such as SharePoint, Excel, PowerPoint and more.
- Check if your organization is Leading or Lagging in terms of AI-engaged time per employee on a weekly basis compared to industry benchmarks.
- Keep track of the week-over-week (WoW) AI interaction trend in Copilot usage for specific applications.

Detecting AI engagement patterns specific to the Microsoft Copilot dashboard

Detect AI engagement patterns of Microsoft Copilot across organizational units and applications by using a heatmap that displays weekly AI engagement time by department and application.

Spot anomalies, such as tools with low activity in high-priority divisions, and simplify decision-making through a colored representation.

Built-in campaigns for monitoring employee perception of AI tools

Ensure all employee devices have Collector version 25.5.1 or later. This guarantees the correct deployment of built-in Nexthink campaigns.

Benchmarking features for Employee experience with AI tools may not be available immediately for launch on October 30th, 2025—to be introduced at a later date.

To display the Employee experience with AI tools in all AI Tool dashboards, Nexthink uses built-in campaigns that you should enable and activate. Once the campaign is activated, allow up to seven days for the data to appear in the AI tool dashboard.

In addition, open the Employee experience campaign responses right-side panel to:

Review each employee comment on Recommended actions individually, including a summary at the top under Insights.
Review each employee comment on Use cases individually, including a summary at the top under Insights.

Refer to the Nexthink Insights - AI Model Card documentation for more information.

The system sends AI tools campaigns with uneditable content adhering to the following standard:

Questions included in each campaign

Each campaign includes these three questions:

How much time has [AI tool] saved you over the past 7 days?
- Options: Time ranges (None, Less than 1h, 1-5h, 6-10h, 11-15h, 15-20h, More than 20h)
How could [AI tool] be more useful to you, and what could we do to help? (Free text input).
What is the last thing you did, or tried to do, with [AI tool]? (Free text input).

Criteria for sending campaigns

Campaigns are sent under the following conditions:

Only employees actively using at least one monitored AI tool receive campaigns.
Each employee receives a campaign at most once every 90 days, and only for one of the tools they use.
A maximum of 5% of the employees receive a campaign on any given day.

If needed, you can modify campaign settings for AI tools to exclude specific users from campaigns or disable the campaign entirely.

Balancing AI adoption

Report adoption results of AI tool solutions to stakeholders and leadership:

Track employee-AI interaction trends to optimize the allocation of licenses/resources.
Measure employee response shifts after conducting suggestions from AI Tools Insights and Recommended actions.
- Validate the effectiveness of targeted communication or training for specific AI tools.
- Track whether walkthroughs or Adopt guides drive measurable improvements.

Ultimately, you should observe tangible value increments as a result of AI tool adoption.

To maintain objectivity in reporting, follow Nexthink dashboards and insights to identify correlations and isolate variables. This way, you avoid over-attributing outcomes to AI tool adoption efforts without sufficient validation.

In addition, support observations with evidence-based frameworks such as statistical correlation measures, e.g., Pearson’s R or controlled comparisons (A/B testing).

RELATED TASKS

Last updated 3 days ago

Was this helpful?