Azure FinOps Essentials

In partnership with

Hi there, and welcome to this week's edition of Azure FinOps Essentials! 🎉

In this edition, we’re diving deep into optimizing your Azure Storage costs. Learn how to select the right storage tiers, manage redundancy options, and implement cost-saving strategies to keep your cloud expenses under control. We’ll also cover the hidden costs associated with advanced features like SFTP, CDN, and Hierarchical namespaces, and how these can add up even if you think you’re only using a small amount of data.

Dive in and start maximizing your Azure Storage savings today!

Cheers,
Michiel

Introduction

Azure Storage is a cornerstone of many cloud architectures, providing versatile options for data storage, ranging from blob storage to file shares. Whether you need to store vast amounts of unstructured data, manage message queues, or host virtual machine disks, Azure Storage offers a solution. Blob storage is ideal for dumping large datasets, offering extreme scalability and ease of use. For simpler data storage needs, Azure Tables and Queues provide lightweight alternatives to more complex services like Cosmos DB or Service Bus, offering ease of consumption with fewer advanced features.

When working with Virtual Machines, their disks are stored in Azure Managed Disks, another form of Azure Storage that ensures reliability and performance. If you require a file share, Azure Files is your go-to, providing SMB or NFS shares that integrate seamlessly with your existing infrastructure. All these options are housed under an Azure Storage Account, which provides a unique namespace to access the services. While creating a storage account is free, you pay based on your actual usage. So, let’s dive into how these costs are structured and how you can optimize them.

Understanding Storage Costs: Operations and Data Access

The cost of your Azure Storage account isn’t just about how much data you store—it’s also heavily influenced by how often and how intensively you access that data. Each operation on your storage account, whether it’s a read, write, or list operation, incurs a charge. This means that even if you store a large amount of data, your costs could be lower than if you store less data but frequently access it.

For example, a write operation might cost around €0.046 per 10,000 operations, while a read operation could be as low as €0.004 per 10,000 operations. These costs can quickly add up if your application frequently interacts with your storage account.

The same principle applies to Azure Queue and Table storage—different types of operations come with different rates. This is why it’s crucial to understand your usage patterns and adjust your code to minimize unnecessary operations. Using the Azure Pricing Calculator can help you estimate these costs and make more informed decisions.

Let’s look at two practical examples:

1. Implementing a Backoff Mechanism for Queues: If you’re using a queue that requires polling (where your application checks in a loop to see if a new message has arrived), consider implementing a backoff mechanism. If there’s no message after a certain number of tries, introduce a delay before the next check. This reduces the frequency of calls and, consequently, your costs. The delay can be removed when messages start arriving again, ensuring that your application remains responsive while minimizing costs.

2. Avoid Unnecessary EnsureTableExists Calls: It’s common to see implementations where the EnsureTableExists function is called before every operation on an Azure Table. While this is a safe approach—ensuring that the table is created if it doesn’t already exist—it incurs an additional operation every time it’s called, which adds to your costs. To optimize, provision your tables as part of your Infrastructure as Code (IaC) rollout, or only perform this check once when the application starts.

By being mindful of how and when you access your storage, you can significantly reduce your Azure Storage costs while maintaining the performance and reliability your applications need.

Influencing Azure Storage Costs: Rate and Resource Optimization

When it comes to managing your Azure Storage costs, there are two main strategies: rate optimization and resource optimization. Both approaches allow you to fine-tune your expenses, ensuring that you get the most value out of your storage investment.

1. Rate Optimization

One way to reduce costs is by applying long-term discounts through reservations. Azure offers 1-year and 3-year reservations for storage, allowing you to lock in lower rates. However, these reservations are typically made in large units—100 TB or even 1 PB of data. This option is ideal for organizations with predictable, large-scale storage needs, allowing you to achieve significant savings by committing to long-term usage.

2. Resource Optimization

The second approach to cost reduction involves optimizing the resources themselves. Here’s how you can make your storage more cost-effective:

Redundancy Levels: Azure provides several redundancy options, each with different cost implications. Choosing the right level of redundancy for your data can have a major impact on your storage expenses:

• Locally Redundant Storage (LRS): This is the most cost-effective option, replicating data synchronously within a single datacenter. While it offers the lowest durability, it’s sufficient for non-critical data.

• Zone-Redundant Storage (ZRS): This option enhances performance and availability by replicating data across up to three physically separate storage clusters within a single region.

• Geo-Redundant Storage (GRS): This option replicates your data to a secondary region far from the primary one, providing greater durability in the event of a regional disaster.

• Read-Access Geo-Redundant Storage (RA-GRS): Similar to GRS, but with the added benefit of allowing read access to the secondary region without a failover event.

Higher redundancy levels offer better data protection and availability but come with increased costs. It’s important to carefully assess your needs—especially for non-production environments where lower redundancy might be acceptable—to avoid overpaying for unnecessary durability.

Access Tiers: Another significant factor in cost optimization is the storage access tier. Azure offers several tiers:

• Hot: The default option, optimized for data that is frequently accessed.

• Cool/Cold: A more cost-effective option for data that is accessed less frequently. Interestingly, access times for Cool and Cold tiers are often similar to Hot, making them attractive for certain workloads.

• Archive: The cheapest option, but with much slower access times, making it suitable for data that is rarely accessed. Be aware that transaction costs for retrieving data from the Archive tier are significantly higher, so use it wisely.

To maximize savings, implement data lifecycle management policies that automatically move data to more cost-effective tiers as it becomes less frequently accessed. For instance, frequently accessed data should remain in the Hot or Cool tier, while older, rarely accessed data can be moved to Archive.

Performance Tiers: Azure also offers a Premium performance tier vs the Standard tier, which uses SSD drives to provide faster data access. While this option improves performance, it also comes with a higher price tag. Evaluate whether the increased speed is necessary for your workload, especially in non-production environments where performance demands may be lower.

The below image shows a comparison between the different options when we store 1 PB of data. Standard, Hot, GRS is the default option, so you see there is a lot of room to change the amount.

By carefully selecting the appropriate redundancy level, access tier, and performance tier for your storage needs, you can significantly reduce your Azure Storage costs while ensuring that your data is stored and accessed efficiently.

Conclusion

Azure Storage offers a range of powerful features, but each comes with its own cost implications. For instance, enabling Hierarchical namespaces can impact transaction rates, adding to your overall expenses. Integrating SFTP might seem like a convenient option, but it comes at a hefty price—around 200 euros per month. Additionally, using your Blob storage as a CDN solution can lead to unexpected egress costs, further increasing your storage bill.

It’s easy to think that a small amount of data will result in minimal costs, but when you start adding up these features, the expenses can quickly escalate. This is why it’s crucial to thoroughly assess your needs and consider the full range of cost factors before enabling additional functionalities in Azure Storage.

By staying informed and carefully planning your storage strategy, you can avoid unexpected charges and ensure that your Azure Storage remains both effective and economical. Remember, small decisions can have a big impact on your cloud budget, so always keep a close eye on the details.

The Daily Newsletter for Intellectually Curious Readers

• We scour 100+ sources daily

• Read by CEOs, scientists, business owners and more

• 3.5 million subscribers

Sign up today!

Thanks for reading this week’s edition. Share with your colleagues and make sure to subscribe to receive more weekly tips. See you next time!

P.S. I have another newsletter about GitHub, Azure, and .NET news. Subscribe as well to keep informed: