An Azure storage account contains all of users' Azure Storage data objects such as blobs, files, queues and tables. The storage account provides a unique namespace for your Azure Storage data that is accessible from anywhere in the world over HTTP or HTTPS.
The data in users' Azure storage account is durable and highly available, secure, and massively scalable.
Consider a scenario where an application listens for messages from a Service Bus Queue or Topic Subscription and writes the messages as Page or Block blobs into a Storage Account Blob.
This continuous writing of messages into the blobs will cause a drastic increase in the number of blobs inside the Storage Account Blob Container. Unlike Service Bus Queue where the messages would be removed from the Queue when the Application receives it, the blobs would retain the content even after the application had done processing it.
In this scenario the blobs older than months is no more required for the Business. Manual deletion of these old blobs is a tedious task. Serverless360's Automated Task capability solves such problems in Storage account resources.
This capability of automating storage account activities is not found in both Azure portal and in the Storage Explorer.
Serverless360 fills this gap with the capability to automate the process of clearing storage data from storage services.
Purging Storage Blobs
Purging blobs can be done by specifying at least one of the following conditions.
- Blob created time
- Blob modified time
- Blob type
- Blob name constraints
The Blob Type is set to All by default.
For Blob Containers with Soft Delete enabled, the blobs will be soft deleted.
Only those blobs that aren't Leased will be deleted.
All the Snapshots of the blobs will also be deleted.
Purge Storage Queue messages
Many enterprises use Storage Queues to communicate between different services. As a part of integrating the services together, it is important to ensure that the message flow across the services is healthy.
Serverless360 provides a capability to automate the process of deleting messages from Storage Queues.
The following configurations will let the action to be performed only on the selected messages.
Message count - The number of messages to be processed.
Message Enqueued date range - Only those messages enqueued at or before the specified date range will be processed on specifying the date range when Customize date range for the configuration checkbox is enabled.
Message Enqueued time span - Only those messages enqueued at or before the specified time will be processed on specifying the time span when Customize date range for the configuration checkbox is enabled.
Purging Storage Files
Azure File Storage has proven useful for many organizations, especially in lifting and shifting applications to the cloud that expect a file share to store file application or user data.
These stored files in Azure file shares might not be of use after a period. Hence removing unwanted content from Azure file share becomes a business need.
Purging files can be done by specifying at least one of the following conditions.
- File modified time
- File name constraints
Files that are deleted will be permanently erased unless snapshot of the file share or backup is taken.
Including directories to purge along with files
When this option is enabled, the configurations specified to purge files (Modified At or Before time and Name constraints) will also be applied to the file share's directories.
When the Automated Task executes and the specified conditions for a directory are met, the directory and all files and subdirectories within it are deleted.
Purging Storage Table entities
Azure Tables are ideal for storing structured, non-relational data. As a result, it is useful in scenarios such as storing a large set of user data for the web application, device information, and service-related metadata.
After a while, it may be necessary to reduce table size by removing old entities.Alternatively, entities that meet a specific condition may need to be deleted.
Entities can be purged by specifying at least one of the following:
- Time stamp
- Filter Query
Deleting based on Timestamp
If the entities are required to be purged based on their added time, users can use the Delete Entities Created At or Before option.
The entities added at or before the specified time will be purged by the Automated task.
The selected Storage account table's entities can also be purged by adding a Filter Query.
Default properties like PartitionKey, RowKey, Timestamp, and User properties can also generate a query filter.
After adding the filter, validate it to continue configuring the Automated task.
The Filter Query conforms to the OData Protocol Specification.
Use DateTime 'ISO date-time formatted string' for the value of type DateTime. Alternately users can also use DateTime 'dd/MM/yyyy HH:mm:ss', which will be converted to ISO format during validation. Provide a local time when using DateTime 'dd/MM/yyyy HH:mm:ss' as the Automated task converts it to UTC before performing validation.
Use guid 'Your 128-bit Guid' for the value of type Guid.
Use suffix L after the value of type Int64.
Use true or false for value of type Boolean.
Use single quotes for the value of type string, and replace any single quote within the string with two consecutive single quotes.
Supported Comparison Operators
|Less than or Equal||le|
|Greater than or Equal||ge|
Supported Binary Operators
Sample Filter Query strings
PartitionKey eq 'PartitionKey' and RowKey eq 'RowKey'
PartitionKey ne 'PartitionKey' or not IsActive
PartitionKey eq 'PartitionKey' and ( UserKey lt 10 or datetime'25/09/2020 10:30:00' )
UserKey eq 'Device''s Info'
Changing Storage Blob Access Tiers
Data stored in the cloud grows immensely. It can be helpful to organize your data based on how frequently it will be accessed and how long it will be retained, as to manage costs for your expanding storage needs.
Azure storage offers different access tiers so that you can store your blob data in the most cost-effective manner based on how it's being used. Azure Storage access tiers include:
Hot tier - An online tier optimized for storing data that is accessed or modified frequently. The Hot tier has the highest storage costs, but the lowest access costs.
Cool tier - An online tier optimized for storing data that is infrequently accessed or modified. Data in the Cool tier should be stored for a minimum of 30 days. The Cool tier has lower storage costs and higher access costs compared to the Hot tier.
Archive tier - An offline tier optimized for storing data that is rarely accessed, and that has flexible latency requirements, on the order of hours. Data in the Archive tier should be stored for a minimum of 180 days.
Thus, Change access tier of a Storage blob is highly beneficial when one needs to automatically change the tier types based on their needs to reduce any billing impacts.
Changing blobs access tier can be done by specifying at least one of the following conditions:
- Blob created time
- Blob modified time
- Blob access tier type
- Blob name constraints
- The snapshots and versions of the blobs of tier type archive cannot be changed to hot or cool tiers.
- The tier cannot be changed for a blob that is being rehydrated
- The tier cannot be changed to archive when either a soft delete of a storage account's blob or container is disabled.
- When a Storage account has its redundancy/replication as GRS, GZRS, and ZRS its blobs/snapshots/versions cannot be changed to tier archive.
- Deleted versions completed count if any will be accounted in the automated task run.
- The Leased blob's snapshot and versions can change access tier only from Serverless360 and its completed task count will also be accounted.
Users can now receive notifications when a Storage Account Automated Task is completed successfully.
The final sections of the Storage Account Automated Task configuration blade includes the notification configuration section, where users can configure the desired configured Notification channels and email address(es) to receive notifications for a group or individually.
All the configured Notification channels will be listed in this section.
Multiple email addresses can also be provided so that a group of users can get notifications and stay in touch.