Retrieving Resource Metrics via the Azure Insights API

 

There are many options for configuring monitoring and alerting for Azure resources. The Azure portal will show some default metrics. It is also possible to enable more advanced or custom diagnostic settings. The links below provide additional details on enabling diagnostics and monitoring for two popular Azure compute resources, Azure Web Apps and Azure Virtual Machines. Visual Studio Application Insights can also be leveraged to monitor the usage and performance of applications.

As previously mentioned, the Azure portal will show some default metrics for many Azure resources. For example, the screenshot below shows the monitoring tile for an Azure Web App, which has been configured to display three key values: Average Response Time, CPU Time, and Requests.

azure_web_app_default_metric_tile

 

The metric values displayed on this tile are not retained indefinitely. For an Azure Web App, the retention policy is as follows:

  • Minute granularity – 24 hour retention
  • Hour granularity – 7 day retention
  • Daily granularity – 30 day retention

By using the Azure Insights API it is possible to programmatically retrieve the available default metric definitions (the type of metric such as CPU Time, Requests, etc.), granularity, and metric values. With the ability to programmatically retrieve the data, comes the ability to save the data in a data store of your choosing. For example, that data could be persisted to Azure SQL Database, DocumentDB, or Azure Data Lake. From there you could perform whatever additional analysis is desired.

It should be noted that the Azure Insights API is not the same as Application Insights.

Besides working with various metric data points, the Insights API allows you to manage things like alerts, autoscale settings, usage quotas, and more. Check out the full list via the Azure Insights REST API Reference documentation.

The remainder of this post will discuss using the Insights API to learn more about the default metrics available for Azure resources.

Investigating Available Metrics via the Insights REST API

There are three basic steps for working with the Insights REST API:

  1. Authenticate the Azure Insights request
  2. Retrieve the available metric definitions
  3. Retrieve the metric values

The first step is to authenticate the Azure Insights API request. As the Azure Insights API is an Azure Resource Manager based API, it requires authentication via Azure Active Directory (Azure AD). The easiest way (in my opinion at least) to set up authentication is by creating an Azure AD service principal and retrieve the authentication (JWT) token. The sample script below demonstrates creating an Azure AD service principle via PowerShell. For a more detailed walkthrough, please reference the guidance at https://azure.microsoft.com/en-us/documentation/articles/resource-group-authenticate-service-principal/#authenticate-service-principal-with-password—powershell. It is also possible to create a service principle via the Azure portal.

Create a Service Principle
  1. # Instructions at https://azure.microsoft.com/en-us/documentation/articles/resource-group-authenticate-service-principal/
  2. $pwd = “[your-service-principle-password]”
  3. $subscriptionId = “[your-azure-subscription-id]”
  4. Login-AzureRmAccount
  5. Select-AzureRmSubscription -SubscriptionId $subscriptionId
  6. $azureAdApplication = New-AzureRmADApplication `
  7.                         -DisplayName “Collier Web Metrics Demo” `
  8.                         -HomePage https://localhost/webmetricdemo” `
  9.                         -IdentifierUris https://localhost/webmetricdemo” `
  10.                         -Password $pwd
  11. New-AzureRmADServicePrincipal -ApplicationId $azureAdApplication.ApplicationId
  12. New-AzureRmRoleAssignment -RoleDefinitionName Reader -ServicePrincipalName $azureAdApplication.ApplicationId
  13. $subscription = Get-AzureRmSubscription -SubscriptionId $subscriptionId
  14. $creds = Get-Credential -UserName $azureAdApplication.ApplicationId -Message “Please use your service principle credentials”
  15. Login-AzureRmAccount -Credential $creds -ServicePrincipal -TenantId $subscription.TenantId

Once the authentication setup step is complete, it is possible to execute queries against the Azure Insights REST API. There are two helpful queries:

  1. List the metric definitions for a resource
  2. Retrieve the metric values

Details on listing the metric definitions for a resource is documented at https://msdn.microsoft.com/en-us/library/azure/dn931939.aspx. For an Azure Web App, the metric definitions should look similar to example screenshot below.

azure_web_app_metric_definitions_with_pointers

 

Once the available metric definitions are known, it is easy to retrieve the required metric values. Use the metric’s name ‘value’ (not the ‘localizedValue’) for any filtering requests (e.g. retrieve the ‘CpuTime’ and ‘Requests’ metric data points). The request / response information for this API call do not appear as an available task at https://msdn.microsoft.com/en-us/library/azure/dn931930.aspx. However, it is possible to do so, and the request URI is very similar to that of listing the metric definitions.

Method Request URI
GET https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/{resource-provider-namespace}/{resource-type}/{resource-name}/metrics?api-version=2014-04-01&$filter={filter}

For example, to retrieve just the Average Response Time and Requests metric data points for an Azure Web App for the 1 hour period from 2016-02-18 20:26:00 to 2016-02-18 21:26:00, with a granularity of 1 minute, the request URI would be as follows:

https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/collierwebapi2016/providers/Microsoft.Web/sites/collierwebapi2016/metrics?api-version=2014-04-01&$filter=%28name.value%20eq%20%27AverageResponseTime%27%20or%20name.value%20eq%20%27Requests%27%29%20and%20timeGrain%20eq%20duration%27PT1M%27%20and%20startTime%20eq%202016-02-18T20%3A26%3A00.0000000Z%20and%20endTime%20eq%202016-02-18T21%3A26%3A00.0000000Z

The result should be similar to that of the screenshot below.

azure_web_app_metric_values

 

Using the REST API is very helpful in terms of understand the available metric definitions, granularity, and related values. This information can be very helpful when using the Azure Insights Management Library.

Retrieving Metrics via the Insights Management Library

Just like working with the REST API, there are three basic steps for working with the Insights Management Library:

  1. Authenticate the Azure Insights request
  2. Retrieve the available metric definitions
  3. Retrieve the metric values

The first step is to authenticate by retrieving the JWT token from Azure AD. Assuming the Azure AD service principle is already configured, retrieving the token can be as simple as shown in the code sample below.

Get Auth Token
  1. private static string GetAccessToken()
  2. {
  3.     var authenticationContext = new AuthenticationContext(string.Format(“https://login.windows.net/{0}”, _tenantId));
  4.     var credential = new ClientCredential(clientId: _applicationId, clientSecret: _applicationPwd);
  5.     var result = authenticationContext.AcquireToken(resource: “https://management.core.windows.net/”, clientCredential: credential);
  6.     if (result == null)
  7.     {
  8.         throw new InvalidOperationException(“Failed to obtain the JWT token”);
  9.     }
  10.     string token = result.AccessToken;
  11.     return token;
  12. }

 

The primary class for working with the Insights API is the InsightsClient. This class exposes functionality to retrieve the available metric definitions and metric values, as seen in the sample code below:

Get Metric Data
  1. private static MetricListResponse GetResourceMetrics(TokenCloudCredentials credentials, string resourceUri, string filter, TimeSpan period, string duration)
  2. {
  3.     var dateTimeFormat = “yyy-MM-ddTHH:mmZ”;
  4.     string start = DateTime.UtcNow.Subtract(period).ToString(dateTimeFormat);
  5.     string end = DateTime.UtcNow.ToString(dateTimeFormat);
  6.     StringBuilder sb = new StringBuilder(filter);
  7.     if (!string.IsNullOrEmpty(filter))
  8.     {
  9.         sb.Append(” and “);
  10.     }
  11.     sb.AppendFormat(“startTime eq {0} and endTime eq {1}”, start, end);
  12.     sb.AppendFormat(” and timeGrain eq duration'{0}'”, duration);
  13.     using (var client = new InsightsClient(credentials))
  14.     {
  15.         return client.MetricOperations.GetMetrics(resourceUri, sb.ToString());
  16.     }
  17. }
  18. private static MetricDefinitionListResponse GetAvailableMetricDefinitions(TokenCloudCredentials credentials, string resourceUri)
  19. {
  20.     using (var client = new InsightsClient(credentials))
  21.     {
  22.         return client.MetricDefinitionOperations.GetMetricDefinitions(resourceUri, null);
  23.     }
  24. }

 

For the above code, the resource URI to use is the full path to the desired Azure resource. For example, to query against an Azure Web App, the resource URI would be:

/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.Web/sites/{site-name}/

 

It is also possible to query the metric data for a classic Azure Virtual Machine – just change the request URI to be appropriate for the classic VM:

/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/microsoft.classiccompute/virtualmachines/{vm-name}/

 

To find the resource URI for a desired resource, one approach is to use the https://resources.azure.com tool. Simply browse to the desired resource and then look at the URI shown, as in the screenshot below.

resource_explorer

 

For the full code sample, please see my GitHub repository available at https://github.com/mcollier/microsoft-azure-insights-demo/.

Thanks to Yossi Dahan’s blog post at https://yossidahan.wordpress.com/2015/02/13/reading-metric-data-from-azure-using-the-azure-insights-library/ for the inspiration.

azure_web_app_metric_value_listing

Deploy a WordPress Azure Web App with an Alternative MySQL Database

I was recently presented with an interesting question about Azure Web Apps, WordPress, and MySQL. While not necessarily a “hard” question, the answer wasn’t as readily available as I first anticipated. I thought I would share my experience here in hopes of helping others.

The Question

How can you deploy a WordPress site using Azure Web Apps that uses a MySQL database instance that is not a ClearDB database available in the Azure subscription?

Background

Normally when you create a WordPress site using Azure Web Apps you are presented with an option to select an existing ClearDB MySQL database, or create a new one. But what if you don’t want to use an existing instance or create a new one? What if you want to use a MySQL database instance deployed to an Azure VM or you have a ClearDB MySQL database that doesn’t show in the Azure Portal (e.g. one of the ClearDB Basic offerings)?

Create_WordPress_ClearDB_Normal

The Answer(s)

Like most technology related questions (or life in general), there are a few ways to solve this challenge. There is the “easy” way, and there is the more powerful, yet slightly more complicated, some would argue the “right” way.

The Easy Way

The easiest approach is to create a WordPress site with Azure Web Apps and select either an existing ClearDB database or create a new ClearDB database. Once the WordPress site is deployed, you can then change the database connection string in the wp-config.php file to be the database you want (e.g. a ClearDB Basic instance or a MySQL instance on an Azure VM).

  1. Let the WordPress site be deployed, but do not complete the installation. In other words, once the site is deployed, browsing to the site’s URL should result in the standard WordPress default installation prompt.
  2. WordPress_Default_Install_1
  3. Open the Kudu console by going to http://your-site-name.scm.azurewebsites.net. If you’re already signed into the Azure Portal, you should proceed through without any authentication challenge. Otherwise you’ll be challenged for your authentication credentials.
  4. Navigate to the Debug console (via the menu on the top). Browse to the \site\wwroot\ directory.
  5. Kudo_wpconfig_1
  6. Edit the wp-config.php file by clicking on the pencil icon to the left of the file name. Doing so will switch to an edit view for the file. Don’t click on the delete icon. . . that’d be a bad thing.
  7. Within the wp-config.php file, change the DB_NAME, DB_USER, DB_PASSWORD, and DB_HOST values to be that of the desired database. Save the file.
  8. WordPress_Default_Install_wpconfig_1
  9. Now reload your site – http://your-site-name.azurewebsites.net. This should load the default WordPress installation page prompting you to complete the WordPress installation.
  10. Complete the installation. This should use the database setting as configured in the wp-config.php file to finish the WordPress installation.
  11. If you created a free ClearDB database to start with, feel free to delete that ClearDB database.

The Alternative

And now the real fun begins! In this alternative approach, an Azure Resource Manager (ARM) template can be used to create the WordPress site on Azure Web Apps and wire up a database of your choosing. To make this happen you will need the ARM template and a MySQL database of your choosing.

To get the ARM template, my first thought was that I could download the template that the Azure Portal is using and simply modify the database connection details to be what I wanted. Wrong. The templates I tried turned out to be a bit more complicated that I wanted. However, they did provide a good start and helped me understand what I needed to do.

If you’re curious, you can get the templates by invoking the PowerShell script below.

# Retrieve all available items
$allGalleryItems = Invoke-WebRequest -Uri "https://gallery.azure.com/Microsoft.Gallery/GalleryItems?api-version=2015-04-01&includePreview=true" | ConvertFrom-Json

# Get all items published by WordPress
$allGalleryItems | Where-Object { $_.PublisherDisplayName -eq "WordPress" }
$allGalleryItems | Where-Object { $_.Identity -eq "WordPress.WordPress.1.0.0" }

# Save default template for all items under directory "C:\Templates"
$allGalleryItems | Foreach-Object {
$path = Join-Path -Path "C:\templates" -ChildPath $_.Identity
New-Item -type Directory -Path $path -Force

$.Artifacts | Where-Object { $.type -eq "template" } | ForEach-Object {
$templatePath = Join-Path -Path $path -ChildPath ( $_.Name + ".json" )

(Invoke-WebRequest -Uri $_.Uri).Content | Out-File -FilePath $templatePath
}
}

(original PowerShell sample from https://github.com/Azure/azure-powershell/issues/1064)

Using the ARM template obtained from the gallery sample as inspiration, I created a new ARM template. You can get the full sample on my GitHub repo at https://github.com/mcollier/AzureWebApp-WordPress-AlternativeDatabase.

"resources": [
{
"apiVersion": "2014-06-01",
"name": "[parameters('hostingPlanName')]",
"type": "Microsoft.Web/serverfarms",
"location": "[resourceGroup().location]",
"tags": {
"displayName": "HostingPlan"
},
"properties": {
"name": "[parameters('hostingPlanName')]",
"sku": "[parameters('sku')]",
"workerSize": "[parameters('workerSize')]",
"numberOfWorkers": 1
}
},
{
"apiVersion": "2014-06-01",
"name": "[variables('webSiteName')]",
"type": "Microsoft.Web/sites",
"location": "[resourceGroup().location]",
"tags": {
"[concat('hidden-related:', resourceGroup().id, '/providers/Microsoft.Web/serverfarms/', parameters('hostingPlanName'))]": "Resource",
"displayName": "Website"
},
"dependsOn": [
"[concat('Microsoft.Web/serverfarms/', parameters('hostingPlanName'))]"
],
"properties": {
"name": "[variables('webSiteName')]",
"serverFarm": "[parameters('hostingPlanName')]"
},
"resources": [
{
"apiVersion": "2014-11-01",
"name": "connectionstrings",
"type": "config",
"dependsOn": [
"[concat('Microsoft.Web/sites/', variables('webSiteName'))]"
],
"properties": {
"defaultConnection": {
"value": "[variables('dbConnectionString')]",
"type": 0
}
}
},
{
"apiVersion": "2014-06-01",
"name": "web",
"type": "config",
"dependsOn": [
"[concat('Microsoft.Web/sites/', variables('webSiteName'))]"
],
"properties": {
"phpVersion": "5.6"
}
},
{
"name": "MSDeploy",
"type": "extensions",
"location": "[resourceGroup().location]",
"apiVersion": "2014-06-01",
"dependsOn": [
"[concat('Microsoft.Web/sites/', variables('webSiteName'))]",
"[concat('Microsoft.Web/Sites/', variables('webSiteName'), '/config/web')]"
],
"tags": {
"displayName": "WordPressDeploy"
},
"properties": {
"packageUri": "https://auxmktplceprod.blob.core.windows.net/packages/wordpress-4.3.1-IIS.zip",
"dbType": "MySQL",
"connectionString": "[variables('dbConnectionString')]",
"setParameters": {
"AppPath": "[variables('webSiteName')]",
"DbServer": "[parameters('databaseServerName')]",
"DbName": "[parameters('databaseName')]",
"DbUsername": "[parameters('databaseUsername')]",
"DbPassword": "[parameters('databasePassword')]",
"DbAdminUsername": "[parameters('databaseUsername')]",
"DbAdminPassword": "[parameters('databasePassword')]"
}
}
}
]
}

The most relevant section is the MSDeploy resource extension (around line 60). It is this extension that deploys WordPress and gets the default database connection string set up. You provide the database server name, database name, database username and database password as input parameters to the ARM template. The ARM template will use those parameters to construct a database connection string in the proper format (set in a variable in the template).

Once the template is created, it can be deployed with a few lines of PowerShell:

#Login-AzureRmAccount

#NOTE - Ensure the correct Azure subscription is current before continuing. View all via Get-AzureRmSubscription -All
#Select-AzureRmSubscription -SubscriptionId "[your-id-goes-here]" -TenantId "[your-azure-ad-tenant-id-goes-here]"

$ResourceGroupName = "dg-wordpress-001"
$ResourceGroupLocation = "East US"
$TemplateFile = "azuredeploy.json"
$TemplateParametersFile = "azuredeploy.parameters.json"

Test-AzureRmResourceGroupDeployment -ResourceGroupName $ResourceGroupName `
-TemplateFile $TemplateFile `
-TemplateParameterFile $TemplateParametersFile `
-Verbose

# Create or update the resource group using the specified template file and template parameters file
New-AzureRmResourceGroup -Name $ResourceGroupName -Location $ResourceGroupLocation -Verbose -Force -ErrorAction Stop

New-AzureRmResourceGroupDeployment -Name ((Get-ChildItem $TemplateFile).BaseName + '-' + ((Get-Date).ToUniversalTime()).ToString('MMdd-HHmm')) `
-ResourceGroupName $ResourceGroupName `
-TemplateFile $TemplateFile `
-TemplateParameterFile $TemplateParametersFile `
-Force -Verbose

The reason I like this approach is that it is very clear what is being deployed. I can customize the template however I like, adding or removing additional resources as needed. Plus, I don’t have to go through that “create a database just to delete it” dance.

For instance, I can envision a version of this ARM template that may optionally set up a MySQL database on an Azure VM. Oh . . . look here, https://azure.microsoft.com/en-us/documentation/templates/wordpress-mysql-replication/. Someone already did mostly just that! That template could be modified to have some options to allow for the creation of a database in a few different configurations. Thanks for saving me some work. Naturally I found this after I went through all the work above. Go figure!  🙂

Lock Down Your Azure Resources

Note: This post is cross-post with http://blogs.msdn.com/b/cloud_solution_architect.

A common ask when working with Azure is “how do I protect my critical resources?” Often this ask is related to protecting those resources from a human doing something they shouldn’t be doing.  There are a few ways to apply varying levels of control to Azure resources: role-based access control (RBAC) and resource locks.

Role-Based Access Control (RBAC)

Using role-based access control, it is possible to restrict control-plane access to various Azure resources. Roles can be assigned at the subscription, resource group, and individual resource scope. A role is associated with specific rights (one or more “action” and “not action” values). There are currently 23 built-in roles.

Leveraging RBAC is a great way to help protect resources in Azure. With the Reader and various Contributor roles (e.g. Virtual Machine Contributor, Storage Account Contributor, SQL DB Contributor, etc.), you can effectively limit the actions that a user can take against a resource. However, even with one of the Contributor roles, it is still possible to delete specific resources. This makes it very easy to accidently delete an item . . . often an item that is not so easy to get back, like an Azure storage account. Oops! Resource locks provide a way to prevent the deletion of resources.

To learn more about RBAC, please check out Neil Mackenzie’s earlier blog post entitled “RBAC and the Azure Resource Manager”, as well as “Role-based access control in the Microsoft Azure portal” on the main Azure site.

Resource Locks

The remainder of this post will focus on a relatively new feature with Azure Resource Manager, resource locks. In order to work with resource locks, you will need to use the Azure PowerShell cmdlets in Azure Resource Manager mode.

Switch-AzureMode AzureResourceManager

You will also need to be in either an Owner or User Access Administrator role for the desired scope, as only Owners and User Access Administrators have permission to write against the Microsoft.Authorization resources, which is the resource provider that handles resource locks.

For more on information on the “actions” and “not actions” associated with specific roles, please see https://azure.microsoft.com/en-us/documentation/articles/role-based-access-control-configure/#built-in-roles.

Create a Resource Lock

A resource lock can be applied at either the resource group or resource scope. As of Azure PowerShell version 0.9.3, there is only one lock level available – CanNotDelete.

To see resource locks in action, let’s use a basic example of creating an Azure Storage account and applying the CanNotDelete resource lock.

$resourceGroup = 'CollierMedia'
$location = 'East US'
$storageAccountName = 'collierfiles2014'

# Create a new resource group.
New-AzureResourceGroup -Name $resourceGroup -Location $location

# Create a new Azure storage account in the new resource group.
New-AzureStorageAccount -ResourceGroupName $resourceGroup -Name $storageAccountName -Location $location -Type Standard_GRS

# Apply a resource lock to the storage account.
New-AzureResourceLock -LockLevel CanNotDelete `
-LockNotes 'No deleting!' `
-LockName 'CollierLock' `
-ResourceName $storageAccountName `
-ResourceType 'Microsoft.Storage/storageAccounts' `
-ResourceGroup $resourceGroup -Verbose

# List all the resource locks in the current subscription.
Get-AzureResourceLock

After executing the above PowerShell commands, you should see the following output result:

powershell - create account and apply lock

Notice in the above screenshot that the “CollierMedia” resource group was created, a new ‘collierfiles2014’ storage account was created in the “CollierMedia” resource group, and a resource lock applied to the storage account. Take particular note of the ResourceId for the lock – that will be helpful when trying to delete a lock.

With the resource lock in place an attempt to delete the storage account fails with an error message indicating the specified resource is locked and needs to be removed before being deleted:

powershell - delete account (masked)

The above example demonstrated creating a resource lock for a specific resource. But what about locking an entire resource group? Placing a resource lock on an entire group could be helpful in situations where you want to ensure no resources in that group are deleted – for example, a resource group containing storage accounts used for Azure Virtual Machines. The example below demonstrates placing a lock on the entire “CollierMedia” resource group.

New-AzureResourceLock -LockLevel CanNotDelete `
-LockNotes 'No deleting!' `
-LockName 'CollierGroupLock' `
-ResourceGroup 'CollierMedia' -Verbose

powershell - lock resource group (masked)

Remove a Resource Lock

One easy way to remove the resource is to use the ResourceId associated with the lock. The ResourceId is returned when creating the lock. It can also be obtained via the Get-AzureResourceLock cmdlet. By default, Get-AzureResourceLock returns a list of all the locks in the current Azure subscription.

powershell - get-azureresourcelock (masked)

View the help (> get-help Get-AzureResourceLock) for additional ways to filter the locks returned.

Use the Remove-AzureResourceLock cmdlet, providing the desired ResourceId, to remove the lock.

Remove-AzureResourceLock -ResourceId '/subscriptions/0bbbc191-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/CollierMedia/providers/Microsoft.Storage/storageAccounts/collierfiles2014/providers/Microsoft.Authorization/locks/CollierLock'

Lock Resources using an Azure Resource Manager Template

Another way to lock a resource is to do so at creation time with an Azure Resource Manager (ARM) template. An ARM template is a JSON-formatted file that describes the desired state of a logically related group of Azure resources. The template could contain the resources needed for a specific web site (i.e. Azure Redis Cache, Azure SQL Database, Azure Storage account and Azure Web App.) or perhaps just a group of related Azure Virtual Machines – the choice is yours. For more information on Azure Resource Manager templates, please follow the links below:

The ARM template below shows an example of creating an Azure storage account and applying the CannotDelete resource lock.

{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"newStorageAccountName": {
"type": "string",
"metadata": {
"description": "Name of the Storage Account"
}

},

"storageAccountType": {
"type": "string",
"defaultValue": "Standard_LRS",
"allowedValues": [
"Standard_LRS",
"Standard_GRS",
"Standard_ZRS"
],

"metadata": {
"description": "Storage Account type"

}

},

"location": {
"type": "string",
"allowedValues": [
"East US",
"West US",
"West Europe",
"East Asia",
"Southeast Asia"
],

"metadata": {
"description": "Location of storage account"
}
}
},

"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"name": "[parameters('newStorageAccountName')]",
"apiVersion": "2015-05-01-preview",
"location": "[parameters('location')]",
"properties": {
"accountType": "[parameters('storageAccountType')]"
},

"resources": [
{
"type": "Microsoft.Storage/storageAccounts/providers/locks",
"name": "[concat(parameters('newStorageAccountName'), '/Microsoft.Authorization/collierLock')]",
"apiVersion": "2015-01-01",
"dependsOn": [ "[concat('Microsoft.Storage/storageAccounts/', parameters('newStorageAccountName'))]" ],
"properties": {
"level": "CannotDelete",
"notes": "Mike's important files - do not delete!"
}
}
]
}
]
}

The corresponding template parameters files is as follows:

{

"location": {

"value": "East US"

},

"newStorageAccountName" : {

"value" : "collierimages2015"

},

"storageAccountType": {

"value": "Standard_GRS"

}

}

To execute this template, the following PowerShell cmdlet can be used:

New-AzureResourceGroupDeployment -ResourceGroupName 'CollierMedia' `
-TemplateFile 'azuredeploy.json' `
-TemplateParameterFile 'azuredeploy.parameters.json' `
-Verbose

Tip: If you would like to validate an ARM template before deploying it, you can use the Test-AzureResourceGroupTemplate cmdlet, such as:

Test-AzureResourceGroupTemplate -ResourceGroupName 'CollierMedia' `
-TemplateFile 'azuredeploy.json' `
-TemplateParameterFile 'azuredeploy.parameters.json' `
-Verbose

Executing the above cmdlets should result in the following output:

powershell - new-azureresourcegroup - create account with locks 2

If we execute the Get-AzureResourceLock cmdlet again, we would notice the resource lock, “collierlock”, successfully applied to just the newly created ‘collierimages2015’ storage account.

powershell - get-azureresourcelock 2 (masked)

One thing to notice about the ARM template for creating the resource lock on the storage account is the name of the nested resource. In the template (see screenshot below as well), you will notice the resource name is a concatenation of the storage account name and the name of the lock – specifically “/Microsoft.Authorization/collierlock”. In this example, “collierlock” is the name of the resource lock.

arm - resource lock name example

The resource name needs to reflect that this is a nested resource. Otherwise, if the resource name was something as simple as “collierlock”, attempting to validate the template with Test-AzureResourceGroupTemplate will result in an error such as the following:

Code    : InvalidTemplate

Message : Deployment template validation failed: ‘The template resource ‘collierlock’ for type ‘Microsoft.Storage/storageAccounts/providers/locks’ at line ’49’ and column ’14’ has incorrect segment lengths. A nested resource type must have identical number of segments as its resource name. A root resource type must have segment length one greater than its resource name.’.

Wrapping It Up

Azure’s Role Based Access Control features, along with resource locks, provide multiple options helping to secure critical Azure resources. RBAC roles provide a great way to limit actions against various types of Azure resources. However, an Owner can still delete a resource. Mistakes happen, and a valuable resource might be accidentally deleted. By leveraging a resource lock on those most crucial Azure resources, you can help to eliminate those “oops . . . did I delete that” moments.

The preceding sections demonstrated how to create a resource lock against an individual storage account, removing the resource lock, and applying a resource lock as part of an Azure Resource Manager template. For another example of using resource locks, please see Alexandre Brisebois’ post at https://alexandrebrisebois.wordpress.com/2015/05/26/protect-mission-critial-azure-virtual-machines-from-accidental-deletion/. In his post, Alexandre demonstrates how to create a resource lock against Azure Virtual Machines.

 

Special thanks to Neil Mackenzie, Charles Lamanna, and Ryan Jones for helping to review this post. Credit to Ryan Jones’ sample on GitHub for information on how to create the resource lock for the storage account.

Audit Logs for Azure Events

Note: This post is cross-post with http://blogs.msdn.com/b/cloud_solution_architect.

A common ask when working with Microsoft Azure is, “how can I view audit logs to determine who made changes to the subscription(s) and the related Azure resources?” With the new Audit Logs feature now available in the Azure Preview Portal (https://portal.azure.com), keeping track of changes becomes easier than ever.

Before getting started, I would like to point out that Dushyant Gill has an excellent post in which he provides one way to answer three very important questions:

  1. Who all has access to my subscriptions?
  2. What access does a specific user have on my subscription?
  3. Who gained (or lost) access to my subscriptions in the past n days?

In Dushyant’s post, he describes a PowerShell module he wrote that makes it easy to generate reports which answer the above questions.

In this post, I will describe one approach for working with the new Azure Audit Logs to determine what changed, and also how to view logs related to the addition or removal of co-administrators. Along with the PowerShell module described in Dushyant’s post, this should allow much more insights into the actions taken against an Azure subscription.

What Changed?

By opening the new Audit Logs blade in the Azure Preview Portal, you can view many of the events that occurred against the subscription.

Audit Logs - Single

The Audit Logs blade will show five key pieces of information for the list of operations:

  1. Operation name
  2. Level: Critical, Error, Warning, or Informational
  3. Status: Succeeded or Failed
  4. Resource
  5. Time

Note 1: Use the Filter action at the top of the Audit Logs blade to modify the operations displayed.

Note 2: Audit logs are available for only the Azure resources available in the Azure Preview Portal and/or the Azure Resource Manager API.

 

Clicking on a specific operation will open additional blades to view more detailed information.

Full Audit Logs

The Detail blade will allow you to view key pieces of information such as who made the change, when the change was made, what was the Azure resource and resource group affected, etc.

Audit Logs - Detail Blade

If the change is related to a role assignment (i.e. adding a user to one of the new Role Based Access Control (RBAC) related roles – Virtual Machine Contributor, Website Contributor, etc.), the Detail blade will show additional information to show the user that was added to the role. At the time of this writing, however, the user and role is displayed as a GUID and not the corresponding friendly name (e.g. email address or role name). It is anticipated this will be cleaned up in a feature update to the Azure Preview Portal. In the meantime, you can leverage a few Azure PowerShell cmdlets to piece together the relevant information.

In the Details blade, have a look at the PROPERTIES value. There are two key pieces of information here: the PrincipalId and the RoleDefinitionId.

Audit Logs - Add User to RBAC Role

For the cmdlets below, you will need to switch to the Azure Resource Manager mode in Azure Powershell by executing the command below.  Azure PowerShell version 0.8.15 is used in this post, although the cmdlets below first appeared in version 0.8.14.

Switch-AzureMode AzureResourceManager

 

To determine the role to which the user was assigned, use the Get-AzureRoleDefintion cmdlet. This cmdlet returns all the role definitions, so you will need to search for the specific definition (the GUID after “roleDefinition”) as indicated the PROPERTIES in the Detail blade.

Get-AzureRoleDefinition | where { ($_.Id -like "*de139f84-1756-47ae-9be6-808fbbe84772" )}

Get-AzureRoleDefinition

To get the user added to the role, use the Get-AzureAdUser cmdlet. For the -ObjectId parameter, use the PrincipalId value from the PROPERTIES in the Detail blade.

Get-AzureADUser -ObjectId b458cfbc-a101-45c1-9575-1ec43aefcc45

Get-AzureAdUser

What about Co-Administrators and Management Certificates?

When it comes to permissions in Azure, there are three approaches that can be used: the co-administrator model, management certificates, and Role Based Access Control (RBAC). The co-administrator model has been the security model in working with Azure for quite some time. People that wanted to work with Azure (assuming they were not the subscription owner) could be granted access to the Azure subscription as a co-administrator. As a co-administrator, they could take any action they desired against any Azure service.  If non-interactive administrative access was needed, a management certificate could be added to the target subscription(s).

Going back to the earlier topic of what change was made and by whom – how can we tell when a co-administrator or management certificate was added to the subscription?

For management certificates, you can use the List Subscription Operations service management API call. There is an Operation Name parameter in the response that should include AddSubscriptionCertificate and RemoveSubscriptionCertificate for the respective action.

AddSubscriptionCertificate

An astute observer of the MSDN documentation for List Subscription Operations will notice there is no corresponding operation for co-administrators. To obtain information about co-administrators being added or removed from a subscription, we can leverage the new audit logs available in the Azure Preview Portal and Azure Resource Manager API. Co-administrators in the Azure Management Portal actually correspond to the Owner role available in the Azure Preview Portal (and thus Azure Resource Manager). Adding or removing a co-administrator via the Azure Management Portal creates an event with Azure Resource Manager that is then logged. However, as of the time of this writing, the addition or removal of co-administrators is not yet reported in the Azure Preview Portal’s Audit Logs.

To get the audit logs related to the addition or remove of co-administrators, you can use the Get-AzureResourceProviderLog PowerShell cmdlet. This cmdlet retrieves the operations associated for a specific Resource Provider – in this case we would want operations against the Microsoft.Authoriztion provider. For example, to view the logs for a certain date range, you could execute a command similar to the following:

Get-AzureResourceProviderLog -ResourceProvider "Microsoft.Authorization" -StartTime "2015-03-10" -EndTime "2015-03-11" -DetailedOutput

A screenshot of the output is below. In the screenshot you’ll notice you can see the adminEmail and adminType for the user added via the Azure Management Portal.

Get-AzureResourceProviderLog - Add CoAdmins

Alternatively, you could get all the authorization events starting from a specific date up until the current time:

Get-AzureResourceProviderLog -ResourceProvider "Microsoft.Authorization" -StartTime "2015-03-11" -DetailedOutput

Get-AzureResourceProviderLog - Remove CoAdmins

It should be noted that there currently does seem to be at least two known bugs related to the shown information:

  1. If the removed co-admin is an Azure AD user (as opposed to a Microsoft Account), the adminEmail is empty
  2. The Caller value is not populating

 

But Wait! There’s More!!

The Get-AzureResourceProviderLog cmdlet shown above is just one of the new audit log cmdlets now available. It is easy to view the additional cmdlets by executing the following command:

get-help get-*Azure*log*
Name Description
Get-AzureSubscriptionIdLog Gets the operations associated with the current subscriptionId
Get-AzureResourceProviderLog Gets the operations associated with a Resource Provider
Get-AzureResourceLog Gets the operations associated with a ResourceId
Get-AzureResourceGroupLog Gets the deployment log for a resource group
Get-AzureCorrelationIdLog Gets the operations associated with a CorrelationId.

 

Summary

The new Audit Logs feature in the Azure Preview Portal, combined with new Azure PowerShell cmdlets, make it easier than ever to have insights into the changes made to an Azure subscription. Together, with the module described in Dushyant Gill’s post, many of the administrative actions taken against an Azure subscription and related resources are now easily viewable.

Please be sure to be using at least version 0.8.14 (that’s when the cmdlets mentioned in this post first appeared) of the Azure PowerShell cmdlets, and switch to use the AzureResourceManager mode for the cmdlets as well.

DevLink 2014 Presentations

Last week I had the privilege to speak at DevLink in Chattanooga, TN.  I had a great time!  It was fun to present.  I was also was able to attend some great sessions.  DevLink is always a top notch event to attend.

I had a full load of presentations – four in three days!  Overall I think the sessions came together very well.  If you attended any of my sessions, I would greatly appreciate any feedback. You can find a copy of all my presentations below and on SlideShare.net

  1. Programming Azure Active Directory
  2. Inside Azure Diagnostics
  3. Automating Your Azure Environment
  4. More Cache for Less Cash

BUILD 2014 Downloads

I don’t know about you, but I’m still trying to catch up on all the Azure hotness Microsoft dropped at BUILD a few weeks ago.  On day 2 of BUILD I was sitting a few rows from the front in the keynote hall at BUILD.  As Scott Guthrie continued with his keynote address, I just kept thinking that the hits just keep coming (in a good way).  Plus, it’s going to take a while to digest and really learn all these great new Azure features.  Exciting!

WP_20140403_08_05_12_Pro

I love the fact that Microsoft makes pretty much all the BUILD sessions available to watch via Channel 9.  This is just a great resource for those that either couldn’t make it to BUILD, or if there, couldn’t attend all the sessions in person (you’d have to clone yourself to make that happen).

I travel a lot and being connected to the internet to watch all the sessions sometimes just isn’t possible.  For me, it is very handy to have the session videos and related PowerPoint slides downloaded to my local PC.  Downloading video and slides for each session would be quite painful.  PowerShell to the rescue!

The PowerShell script below is one that I’ve been using for a few years now to download material from BUILD, TechEd, and other major Microsoft events that have content hosted on Channel 9.  I made a few recent tweaks to handle some quirks with the way BUILD 2014 slides were being shared. There’s still some room for improvement, but for the most part it gets the job done.  And that job is downloading all the Azure related sessions and supporting videos to my local computer.

[gist https://gist.github.com/mcollier/10097074]

 

Getting the Windows Azure PowerShell Cmdlets

I’m a big fan of using the Windows Azure PowerShell cmdlets for managing my Windows Azure resources.  They’re incredibly helpful, especially when I need to repeatedly take care of multiple tasks.

There are two ways to get the Windows Azure PowerShell cmdlets – the heavy way and the light way.

Get All the Things!

I would assume most people get the Windows Azure PowerShell cmdlets by taking the path of least resistance.  They go to the Windows Azure downloads page, click the link to “Install” the cmdlet, and let the Web Platform Installer (WebPI) do its thing.  That’s all fine, but it does a lot.  What’s a lot?  Let’s see . . .

Azure PowerShell Web PI

Trying to install the cmdlets via WebPI will also attempt to install a bunch of other software you might not expect, such as:

  • Windows Azure Emulator
  • Microsoft SQL Express LocalDB Edition
  • IIS 8.0 Express
  • Windows Azure Storage Tools
  • Windows Azure Authoring Tools
  • Windows Azure SDK

If you’re an IT Pro / systems admin type of person, that’s probably more than you would expect, or need.

Get Just the One Thing!

If you want just the Windows Azure PowerShell cmdlets, the path of absolute least resistance is to download the latest release from GitHub – https://github.com/WindowsAzure/azure-sdk-tools/releases.  From here you can find links to download via Web PI (we already know what that will bring with it), or get the MSI by itself.

powershell_github

Download and run the MSI and you start off with something pretty familiar and basic.

powershell_msi