Skip to content

Azure File Sync – Optimizing Your File Server and Storage

08/28/2019

Two more applications consume storage in IT environments:

  • Database – Because they contain analytical and indexed data we can use drill down techniques to separate analytical data from summary data for easier access and cost optimization.
  • File Server – Over the years, companies accumulate thousands of files, which is expensive and rarely clustered or tiered.

Tiering: Technology where data is separated according to performance rules on more expensive or cheaper disks.For example, underused files are on SATA disks, files with occasional access on SAS disks, and files that are accessed daily on SSD disks.

Let’s cover how to use Azure File Sync to create a tiering of data on a File Server to allow most accessed files to be stored locally and the oldest to cloud only.

Frequent Scenarios

The first scenario is to decrease the total size of space occupied by old files.

In this case we use the desired file date and free space settings to decrease the disk space that File Server occupies, freeing up for use with other needs.

The second scenario is distributed file server, where in each branch of the company you need to have a server to access the data.

In this example all servers replicate the same folder, which does not create local saturation problems, since the cache is only of recent files and controlled by the desired percentage of free space to be kept.

Azure File Sync Components
  1. Storage Account – A virtual storage where data will be stored
  2. Storage Account File Share – Folder within Storage Account to receive files to upload
  3. Azure File Sync Service at Market Place – Is the service and should be enabled, unlike other native services.However, despite being at Market Place AFS does not have a cost, it is just the inclusion of a service
  4. File Sync Service – Azure dashboard service where we can create groups, add servers and configure storage
  5. Registered Services (servers) – These are the servers that will be synchronized, where the files are stored and will be cached
  6. Sync Group – Form the list of servers that will receive the copy of the files to be copied and give access to the files in any location.
Creating a storage

This is the first and well-known step for those who already use Azure, since we need storage for everything.

armazenamento

To use AFS you do not need any additional configuration, you will be able to choose which region, storage type, and replication that best applies to your environment. Obviously some things need to be taken into account:

  • Account type involves maximum performance and will affect both download and upload when users use files
  • Replication is important if you will have servers in multiple locations / countries.
  • Hot or Cold layer involves performance directly as well as cost, as access is very slow on Cold disks and would not recommend for a solution like this.

Following is the need to create File Share where the files will go when synchronized, and the concept is the same as a common server:

compartilhamento

When synchronized, the files will appear first in the Synchronization folder and then in the main folder as we can see below.

syncstaging

Files Sync

Remember that the two screens above refer to the synchronization already completed, the first to see the files being copied and the second when the first synchronization has already finished.

Enabling Azure File Sync

Search the Marketplace for Azure File Sync or Azure Sync Service:

mktplace

mktplace-2

At this point you can choose to use an existing or new Resource Group, no matter which Resource Group Storage was created in, as it may have several other services assigned.

Creating the synchronization service

The creation of the sync group is quite simple, just indicate the signature, storage and shared folder defined earlier.

Servico

grupo sincronizacao

Registering File Servers

You may refer servers:

  • New servers that do not have files and include them in an already synchronized group so that it caches files that are already in the Azure Storage shared folder
  • Server with data where content will be copied to Azure and added

The first step is to install Azure PowerShell libraries (AZ) on the server, which can be done by following the steps on https://docs.microsoft.com/en-us/powershell/azure/install-az-ps?view=azps-2.6.0&wt.mc_id=4029139

Once you have Azure CLI installed, download and install the Sync Agent which is very simple to do.

AZFAgente

registerserver

After that you will be able to see the server in the Azure dashboard:

serverregistrado

No further configuration or configuration is required at this step as it is a simple agent operation.

Creating the Endpoint (Cache Servers)

This is where we really create the service and watch the magic happen!

Entering the sync group we created earlier and using the Add Endpoint or Add Endpoint option to include the server in the group we created.

Extremidade

Let’s look at the options that are listed:

  1. Path – This is the directory we want to be synchronized, remembering that if it is empty for an existing group it will download content as it is used. If it is a server that already contains files, these will be uploaded to Azure.
    Important: It is not possible to use the root drive (C :), but one part dial because of the system files.
  2. Free Volume Percentage – We do not define how much to use for caching, but how much space on the volume should be free. It may seem like an inverted calculation but it is not because of other files that the same disk contains. For example, if the volume is 100GB and contains other files totaling 40GB and we define that we want to leave 50% of the disk free, only 10GB will be used by cache (50% of 100GB = 50GB always free) and as the use of other files increases. that are not synchronized, less will have room for cache.
    Tip: Because of this difficulty, prefer to use a dedicated volume to make File Sync
  3. Cache only files accessed or modified ax di as – We saw that we have the option to preserve a percentage of the disk. But what if old files take up a lot of space won’t do much good. In this case of my example any file older than 60 days will automatically go to Azure and will be deleted on the server disk, gaining free space even if the cache percentage is still available.

Painel

At the end of this configuration it is possible to follow the synchronization by clicking on the server:

Server sync

Once synchronized, we can use the metrics panels below the screen to create alerts when errors or distortions occur:

Metricas

In my example I can use a rule that if the number of synced files is greater than 100 for upload within 15 minutes could be a mass change caused by improper copying or even malware .

Advertisements
Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: