# Exporter for Azure Data Lake

{% hint style="warning" %}
This documentation references external sources. Nexthink does not have control over the accuracy of third-party documentation, nor any external updates or changes that might create inconsistencies with the information presented on this page. Please report any errors or inconsistencies to [Nexthink Support](https://support.nexthink.com/).
{% endhint %}

Data Export allows you to export Nexthink data insights into Azure Data Lake via CSV files, comma separated and in UTF-8 format.

Configure Azure Data Lake Storage Gen2 data destinations to store data and create a Data Export in Nexthink web interface to distribute it.

## Prerequisites <a href="#exporterforazuredatalake-prerequisites" id="exporterforazuredatalake-prerequisites"></a>

Before setting up the date exporter in the Nexthink web interface, you must first:

1. [Create a Storage Account in Azure Portal](#exporterforazuredatalake-creatingastorageaccountinazureportalstorage_account)
2. [Create a data lake Container within a Storage Account](#exporterforazuredatalake-creatingadatalakecontainerwithinastorageaccountcointainerstorageaccount)
3. [Register an application in Azure Portal](#exporterforazuredatalake-registeringanapplicationinazureportalregister_application)

### Creating a Storage Account in Azure Portal <a href="#exporterforazuredatalake-creatingastorageaccountinazureportalstorage_account" id="exporterforazuredatalake-creatingastorageaccountinazureportalstorage_account"></a>

Log in to the [Azure](https://portal.azure.com/) portal:

To create a file system using the general-purpose v2 storage account (not a data lake storage gen1) in the Azure portal, follow these steps:

1. In the Azure portal menu, select All services.
2. In the list of resources, type *Storage Accounts*. As you begin typing, the list filters the content based on your input.
3. Select **Storage Accounts**.
4. Select **Add** in the Storage Accounts window.
5. Select the subscription for which you want to create the storage account.
6. Select **Create new** under the **Resource group** field. Enter the name of your new resource group. If a Resource group already exists, select it from the dropdown list.

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-4cb55d654febf51b60a4881224564e4374d2ea5f%2Fimage-20220830-082528.png?alt=media" alt=""><figcaption></figcaption></figure>

7. Enter the name of your storage account. The name you choose must be unique across Azure. The name also must be between 3 and 24 characters in length and can include numbers and lowercase letters only.
8. Select a location for your storage account or use the default location.
9. Fill in the information for the rest of the tabs, **Advanced**, **Networking**, **Data Protection**, **Encryption and Tags.**
   * **Disable** *Soft delete for blobs* in the Data Protection tab

     <figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-80617baf390d1d9aa5e1df358248238a9230857e%2Fscreenshot-2022-10-26-at-09-42-40-20221026-074321.png?alt=media" alt=""><figcaption></figcaption></figure>
   * **Enable** *Hierarchical namespace* in the Advanced tab

     <figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-2a4f61049ed703a321e1ba14d256f1c5cae308e1%2Fscreenshot-2022-10-17-at-08-40-19.png?alt=media" alt=""><figcaption></figcaption></figure>
10. Select **Review + Create** to review your storage account settings and create the account.

### Creating a data lake Container within a Storage Account <a href="#exporterforazuredatalake-creatingadatalakecontainerwithinastorageaccountcointainerstorageaccount" id="exporterforazuredatalake-creatingadatalakecontainerwithinastorageaccountcointainerstorageaccount"></a>

1. Locate your newly created storage account under **Storage accounts**.
2. Select the storage account you want to use.
   * You must create a new container.
3. Select **Containers**, add a new container and enter a meaningful name for it, for example, openbridge-data-lake.
4. Make sure access is set to **Private (no anonymous access)**.
5. Click on **Create.**

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-bbaa3495859f1ef5a0fc21f035ddc479e8da2b35%2Fscreenshot-2022-08-30-at-10-33-59.png?alt=media" alt=""><figcaption></figcaption></figure>

### Registering an application in Azure Portal <a href="#exporterforazuredatalake-registeringanapplicationinazureportalregister_application" id="exporterforazuredatalake-registeringanapplicationinazureportalregister_application"></a>

Register an application with the [Microsoft identity platform](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app) and apply the valid role assignment for access.

1. Register a new application in the Azure portal.
2. Select the account type based on your business requirements.

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-603ae77ac221ac6cbf112633eb09d563adf008cc%2Fimage-20220830-083831.png?alt=media" alt=""><figcaption></figcaption></figure>

3. Assign the **Storage Blob Data Owner** or **Storage Blob Data Contributor** role to the service principal, which grants the service principal full access to blob data rights. Assign other blob data roles according to your business requirements. For the details of built-in role permissions, please refer to the [Azure built-in roles](https://docs.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#storage-blob-data-owner) documentation on the Microsoft website.

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-b0870dfc7173456c48d4918e8eda4c4015721130%2Fexporterforazuredatalake-1.png?alt=media" alt="ExporterforAzureDataLake-1.png" width="760"><figcaption></figcaption></figure>

## Configuring Azure DL connector credentials in Nexthink <a href="#exporterforazuredatalake-configuringazuredlconnectorcredentialsinnexthink" id="exporterforazuredatalake-configuringazuredlconnectorcredentialsinnexthink"></a>

1. Select **Administration** from the main menu.
2. Click on **Connector credentials** from the Integrations section of the navigation menu.

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-9f8816b03ccbd0153ccae55a02526595c93d333a%2Fdata-1663234400.png?alt=media" alt="Accessing Connector credentials"><figcaption></figcaption></figure>

3. Click on the **New credential** button located in the top-right of the Credentials page.
4. Fill in the **Name**.
5. Select **HTTPS** as the protocol.
6. Select **OAuth 2.0 - Client Credentials** as the authorization mechanism.
7. Fill in the credential values with the Storage and Application information from the Azure portal. See the image and values listed below.

{% hint style="info" %}
Refer to the [Connector credentials](https://docs.nexthink.com/platform/configuring_nexthink/bringing-data-into-your-nexthink-instance/integrating-nexthink-with-third-party-tools/outbound-connectors/connector-credentials) documentation for more information.
{% endhint %}

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-a5e8f881354b7f00e9d0335a46b70670b7f428d3%2Fexporterforazuredatalake-credentials-2.png?alt=media" alt="ExporterforAzureDataLake-credentials-2.png"><figcaption></figcaption></figure>

* **URL address:** `https://$STORAGE_ACCOUNT_NAME.dfs.core.windows.net`
  * The `$STORAGE_ACCOUNT_NAME` name of the Storage created in [Step 1](#exporterforazuredatalake-creatingastorageaccountinazureportalstorage_account).
* **Access token URL:** `https://login.microsoftonline.com/$TENANT_ID/oauth2/v2.0/token`\
  `$TENANT_ID`: Directory (tenant) ID of the application created in [Step 3](#exporterforazuredatalake-registeringanapplicationinazureportalregister_application).
  * In the Overview section, copy the Directory (tenant) ID.

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-1fc233d92c7e5aaf1a1c4eb1ebd2f731c69e8faa%2Fimage-20220830-094211.png?alt=media" alt=""><figcaption></figcaption></figure>

* **Client ID:** Select Application (client) ID from the Overview section of the application created in [Step 3](#exporterforazuredatalake-registeringanapplicationinazureportalregister_application).

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-65a88a792c2fc58e38a2280eac3417e666c763a3%2Fimage-20220830-093723.png?alt=media" alt=""><figcaption></figcaption></figure>

* **Client secret:** Value taken from the **Certificates & secrets** section of the application created in [Step 3](#exporterforazuredatalake-registeringanapplicationinazureportalregister_application).

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-ac12d67377d723bee629bee268a5ebc5d497b349%2Fexporterforazuredatalake-6.png?alt=media" alt="ExporterforAzureDataLake-6.png" width="760"><figcaption></figcaption></figure>

* **Scope:** `https://storage.azure.com/.default`

## Creating an Azure DL data exporter in Nexthink <a href="#exporterforazuredatalake-creatinganazuredldataexporterinnexthink" id="exporterforazuredatalake-creatinganazuredldataexporterinnexthink"></a>

You must have administrative permissions to configure the data exporter. Refer to the [Adding users](https://docs.nexthink.com/platform/user-guide/administration/account-management) documentation for more information about user role.

To access Data Export:

1. Select **Administration** from the main menu.
2. Select **Outbound connectors** from the **Integrations** section of the navigation panel.

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-f8d5c249bc96fba2535d3d41effa87128cd8ba55%2Fdata-1663234798.png?alt=media" alt="Accessing outbound connectors"><figcaption></figcaption></figure>

3. Select **Data Exporter** from the list of Outbound connectors. When you access the Data exporter page for the first time, there are no elements on the page. Once you create a data exporter, it is listed on the page along with the total number of created data exporters.
4. Click on the **New exporter** button located in the upper-right corner of the page to create a new data exporter.

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-c5ce25542a0111bb1569cd76633fed6c59c16166%2Fnewdataexportersbutton.png?alt=media" alt="NewDataExportersButton.png" width="760"><figcaption></figcaption></figure>

5. Select the **Azure DL** data exporter type.

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-a7ffbc5c281f63ee8649d4b06d441db126b4d0d7%2Fdataexportertypes.png?alt=media" alt="DataExporterTypes.png" width="760"><figcaption></figcaption></figure>

### General tab <a href="#exporterforazuredatalake-generaltab" id="exporterforazuredatalake-generaltab"></a>

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-e310b8eb5b4591b8ad2987e73485e4735f24c544%2Fazuredl_general_tab.png?alt=media" alt="AzureDL_general_tab.png" width="760"><figcaption></figcaption></figure>

* **Name:** enter a meaningful name for the data exporter.
* **Description:** enter a meaningful description of the goal of the data exporter.
* **Active:** switch on the toggle to enable the exporter.
* **Credentials:** define credentials from the third-party tool that the data exporter sends the data to. Refer to [Connector credentials](https://docs.nexthink.com/platform/configuring_nexthink/bringing-data-into-your-nexthink-instance/integrating-nexthink-with-third-party-tools/outbound-connectors/connector-credentials) for more information.
* **Container:** enter the name of the container in Azure Data Lake to push the data to.
* **Maximum file size:** define the maximum file size generated by Data Export. If the data set from a specific **NQL query** is larger than the specified file size, it splits it into several separate files.

### Data tab <a href="#exporterforazuredatalake-datatab" id="exporterforazuredatalake-datatab"></a>

<figure><img src="https://268444917-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxJSUDk9NTtCHYPG5EWs3%2Fuploads%2Fgit-blob-d25d47c9e540e69a9396a76c13bb6bd2eacc2de4%2Fazuredl_data_tab.png?alt=media" alt="AzureDL_data_tab.png" width="760"><figcaption></figcaption></figure>

* **Scheduling frequency:** define how often the system executes the NQL query and exports data. The available options are:
  * **Hourly:** The system triggers the data export data based on the value selected in the drop-down menu. For instance, every 1h, 2h, 3h, 4h, 6h or 12h.
  * **Daily:** The system triggers the data export every day at 00:00 of the timezone where the Nexthink cloud instance is located.
  * **Weekly:** The system triggers the data export weekly, on the selected day at 00:00 of the timezone where the Nexthink cloud instance is located.

For **Hourly** and **Daily** scheduling frequencies, the system waits 20 minutes to execute the data exporter to allow the previous bucket to close properly for data completeness.

{% hint style="info" %}
You must select a value for the **Recurrence** option because the system does not generate a default value and does not indicate that the value is missing during the validation process.
{% endhint %}

* **NQL query:** define the data exported from Nexthink web interface into the destination using an NQL query.
* **File name:** Enter the filename created in the destination. Underscore is the only special character supported, for example: *IT\_lake\_exporter*.
* **Directory (optional):** define the directory within the container credentials from the third-party tool that the data is exported to. If the directory doesn’t exist, the system will automatically create it.

### Testing the Azure DL data exporter <a href="#exporterforazuredatalake-testingtheazuredldataexporter" id="exporterforazuredatalake-testingtheazuredldataexporter"></a>

Click the **Tests load up to 20 records** button to validate the connection to Azure Data Lake before saving the configuration.

* If the NQL query and the connection are valid, a message appears indicating that the query results has been successfully delivered.
* If the NQL query or the connection is invalid, a message appears informing about the error details.

{% hint style="info" %}
Refer to the [Managing data exporters](https://docs.nexthink.com/platform/configuring_nexthink/bringing-data-into-your-nexthink-instance/integrating-nexthink-with-third-party-tools/outbound-connectors/data-exporters/managing-data-exporters) documentation to learn more about data exporters creation, editing, deleting and disabling.
{% endhint %}

## Limits <a href="#httpapiexporter-f.a.q" id="httpapiexporter-f.a.q"></a>

Refer to the [Nexthink Infinity thresholds and limits overview](https://edocs.nexthink.com/nexthink-infinity/infinity-specifications/nexthink-infinity-default-thresholds-overview) for the list of the most up-to-date limits.
