How can I integrate data sources that are secured via private endpoints into Fabric? How do I deal with Azure Data Lakes behind a firewall? This blog post shows the possibilities which Fabric Nativ offers
Considerations of network security need to cover incoming as well as outgoing data transfer.
Inbound data traffic describes access to Fabric itself (e.g. invoking app.fabric.microsoft.com). It is possible to integrate Fabric into a network via private links. This makes Fabric available exclusively via the internal network. Networks from the Internet are blocked.
Two parameters are used in the admin portal for this purpose: Azure Private Link and Block Public Internet Access.
Private links enable services to be delivered across private virtual networks (Vnet) without the need to connect them via peering.
"Block Public Internet Access" deactivates all traffic via the Internet. It is important to ensure that a private link is configured correctly before the setting is enabled. Otherwise there is a risk of locking oneself out of Fabric.
For outbound data traffic, access from Fabric to external data sources is of central importance. Different scenarios are possible here:
Scenario 1: Azure Storage / Data Lake Gen 2 behind firewall
It is possible to whitelist all Fabric workspaces of the tenant or individual ones. Though all the tenant's workspaces can be specified, this is not recommended because the feature might be discontinued in future.
Individual workspaces can also be specified via the ARM template of the storage account / data lake:
"resourceAccessRules": [
{ "tenantId": " df96360b-9e69-4951-92da-f418a97a85eb",
"resourceId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/Fabric/providers/Microsoft.Fabric/workspaces/b2788a72-eef5-4258-a609-9b1c3e454624"
}
]
Here, however, only „b2788a72-eef5-4258-a609-9b1c3e454624“ is to be replaced with the own workspace ID. This can be found in the Fabric portal when invoking the workspace, and is indicated as part of the URL.
In addition to the network, there is also identity as a security parameter 1. Therefore, the appropriate permissions still have to be set. Two access models are available here: Firstly, access-control lists and secondly, role-based access-control assignments (RBAC). For RBAC assignments, it is important to keep in mind that there are dedicated roles for data lakes and storage accounts (e.g. Storage Blob Data Reader).
Restrictions at a glance:
Scenario 2: Azure PaaS resources integrated into the network via private endpoints are to be accessed by Fabric.
This is not the case if Anonymus Blob Access is activated (only recommended for fewer use cases, as authentication is thus bypassed).
Here, it is possible to work in Fabric via managed private endpoints and managed Vnets
This must then be approved in the Azure portal:
After that, it is possible to access the data sources with notebooks and Spark job definitions via the private endpoints. Access via pipelines is currently not possible, however. Spark notebooks
require CPU power, for which a cluster of virtual machines is created in the background. By default, the machines share a standard network. In this scenario, an isolated, managed network is created during the first run.
Restrictions at a glance:
Another scenario mentioned here for the sake of completeness is connectivity to on-premises data sources. The on-premises data gateway can be used to access these data sources directly.
Ultimately, it can be said that the private-endpoint feature often being discussed currently is only relevant if a data source existent in Azure is to be connected. In this case, one should be aware that the data source needs to be loaded via notebooks / Spark job definitions. Access via pipelines is not possible.
Fabric itself or the capacity cannot be integrated into a network via a simple private endpoint. A more complex setup is needed for this purpose. A private link must be used, and the tenant settings for this link and/or the Block Public Internet Access setting must be activated.
However, companies wanting to avoid a use of network integration can also employ other security measures, as described at the beginning of the article. In the end: Fabric security means more than the use of private endpoints.
Looking to become more data-driven, optimize processes, or leverage cutting-edge technologies? Our blog provides valuable insights – but the best way to tackle your specific challenges is through a direct conversation.
Let’s talk – our experts are just one click away!
Managing Director
Do you want to replace the IoT core with a multi-cloud solution and utilise the benefits of other IoT services from Azure or Amazon Web Services? Then get in touch with us and we will support you in the implementation with our expertise and the b.telligent partner network.
You want to manage your infrastructure with Terraform, but then it happens – manual changes are made, and you need to find a solution. How to handle this depends on the specific case.
One of Terraform’s greatest strengths is its ability to handle changes made outside its managed resources. The keywords are: data, import, removed, ignore_changes, lock, variables.
Many security considerations involving Azure revolve primarily around network security. Other important security aspects to be considered in the context of Microsoft Fabric are indicated below.
As stated in part one of this blog series on the reference architecture for our cloud data platform, we will share and describe different parts of this model, and then translate it for the three major cloud providers – Google Cloud Platform, AWS, and Azure. In case you just came across this blog post before seeing the first one about the ingestion part of our model, you can still read it here first. For all others, we’ll start by looking at the data lake part of the b.telligent reference architecture before diving deeper into analytics and DWH in part 3.
No previous post
No next post
