My New Azure Book

Last week Microsoft Press released the latest update to my book, Microsoft Azure Essentials: Fundamentals of Azure, Second Edition.   As the name indicates, this is an update to Microsoft Azure Essentials: Fundamental of Azure released in February 2015. I worked on both editions with my friend Robin Shahan.

The first edition of the book was relatively popular, with over 200,000 copies either downloaded or distributed in print. Thank you for the amazing support! I’m very happy so many people found the book to be helpful.fundamentalsofazure2e-thumbnail

When the Microsoft Press editor that worked on the first book initially approached me about working on an update, I was honestly a bit hesitant. Writing a technical book is a lot of work . . . especially a book on a platform such as Azure that seems to be constantly evolving. However, after many discussions with Robin, and my wife (getting spousal approval was very important as this effort would require many nights and weekends away from my family, which added a new baby to the party earlier this year), I decided to write the update. Thankfully Robin agreed to join in on the fun too. She’s a lot of fun to work with!

My thinking going into this process was there would be a few significant updates, but mostly it would be changing a few screenshots, updating a product name or two (i.e. Azure Web Sites became Azure Web Apps), and maybe including a bit of content around new services like Azure Service Fabric. It wouldn’t be that much work. Wrong!

Azure is a fast-moving platform. In the nearly 13 months since the first edition of the book was released to when I got started on the second edition, many things had changed. The second edition contains quite a few significant updates. The sections on Azure Web Apps, Azure Virtual Machines, Virtual Networks, Storage, and management tooling all received major edits. Key concepts such as Azure Resource Manager, use of the (new) Azure Portal, and the (almost entire) removal of Azure Cloud Services also received increased focused in the second edition. We also added a new chapter on Additional Azure Services which provides a brief introduction to several other Azure services Robin and I felt were important to understand. There are so many valuable services in the Azure platform, but we couldn’t discuss all of them in the book (or else we still might be working on the book, and it would be huge). These are services we personally felt were of significance for the broad majority of people. (Robin adds more info on the changes in her comment here.)

I would be remiss if I didn’t once again thank the Azure experts that volunteered their precious time to help review the book. Many of these people helped on the first edition too. Their expertise and honest feedback was critical in writing this book. Thank you!

I hope you find the second edition of the book to be a valuable asset to your journey to the public cloud with Microsoft Azure!

 

 

Work on Your ARM Strength

Last month I had the privilege to speak at one of the many excellent technology conferences in the Midwest, StirTrek. It’s always fun to speak, and attend, StirTrek. Not only are the sessions great, but afterwards there is a showing of one of the summer’s biggest movies. This year the movie was Captain America: Civil War.

My session this year was “Work on Your ARM Strength”. The goal with the session was to provide some guidance on how to write Azure Resource Manager (ARM) templates. ARM templates are a key part of working with Azure, and it is important to understand how to write, debug, and deploy them.

StirTrek did an excellent job in recording this year’s sessions. You can view all the session on StirTrek’s YouTube channel. You can watch my “Work on Your ARM Strength” session here, or via the embedded video below. Enjoy!

Retrieving Resource Metrics via the Azure Insights API

 

There are many options for configuring monitoring and alerting for Azure resources. The Azure portal will show some default metrics. It is also possible to enable more advanced or custom diagnostic settings. The links below provide additional details on enabling diagnostics and monitoring for two popular Azure compute resources, Azure Web Apps and Azure Virtual Machines. Visual Studio Application Insights can also be leveraged to monitor the usage and performance of applications.

As previously mentioned, the Azure portal will show some default metrics for many Azure resources. For example, the screenshot below shows the monitoring tile for an Azure Web App, which has been configured to display three key values: Average Response Time, CPU Time, and Requests.

azure_web_app_default_metric_tile

 

The metric values displayed on this tile are not retained indefinitely. For an Azure Web App, the retention policy is as follows:

  • Minute granularity – 24 hour retention
  • Hour granularity – 7 day retention
  • Daily granularity – 30 day retention

By using the Azure Insights API it is possible to programmatically retrieve the available default metric definitions (the type of metric such as CPU Time, Requests, etc.), granularity, and metric values. With the ability to programmatically retrieve the data, comes the ability to save the data in a data store of your choosing. For example, that data could be persisted to Azure SQL Database, DocumentDB, or Azure Data Lake. From there you could perform whatever additional analysis is desired.

It should be noted that the Azure Insights API is not the same as Application Insights.

Besides working with various metric data points, the Insights API allows you to manage things like alerts, autoscale settings, usage quotas, and more. Check out the full list via the Azure Insights REST API Reference documentation.

The remainder of this post will discuss using the Insights API to learn more about the default metrics available for Azure resources.

Investigating Available Metrics via the Insights REST API

There are three basic steps for working with the Insights REST API:

  1. Authenticate the Azure Insights request
  2. Retrieve the available metric definitions
  3. Retrieve the metric values

The first step is to authenticate the Azure Insights API request. As the Azure Insights API is an Azure Resource Manager based API, it requires authentication via Azure Active Directory (Azure AD). The easiest way (in my opinion at least) to set up authentication is by creating an Azure AD service principal and retrieve the authentication (JWT) token. The sample script below demonstrates creating an Azure AD service principle via PowerShell. For a more detailed walkthrough, please reference the guidance at https://azure.microsoft.com/en-us/documentation/articles/resource-group-authenticate-service-principal/#authenticate-service-principal-with-password—powershell. It is also possible to create a service principle via the Azure portal.

Create a Service Principle
  1. # Instructions at https://azure.microsoft.com/en-us/documentation/articles/resource-group-authenticate-service-principal/
  2. $pwd = “[your-service-principle-password]”
  3. $subscriptionId = “[your-azure-subscription-id]”
  4. Login-AzureRmAccount
  5. Select-AzureRmSubscription -SubscriptionId $subscriptionId
  6. $azureAdApplication = New-AzureRmADApplication `
  7.                         -DisplayName “Collier Web Metrics Demo” `
  8.                         -HomePage https://localhost/webmetricdemo” `
  9.                         -IdentifierUris https://localhost/webmetricdemo” `
  10.                         -Password $pwd
  11. New-AzureRmADServicePrincipal -ApplicationId $azureAdApplication.ApplicationId
  12. New-AzureRmRoleAssignment -RoleDefinitionName Reader -ServicePrincipalName $azureAdApplication.ApplicationId
  13. $subscription = Get-AzureRmSubscription -SubscriptionId $subscriptionId
  14. $creds = Get-Credential -UserName $azureAdApplication.ApplicationId -Message “Please use your service principle credentials”
  15. Login-AzureRmAccount -Credential $creds -ServicePrincipal -TenantId $subscription.TenantId

Once the authentication setup step is complete, it is possible to execute queries against the Azure Insights REST API. There are two helpful queries:

  1. List the metric definitions for a resource
  2. Retrieve the metric values

Details on listing the metric definitions for a resource is documented at https://msdn.microsoft.com/en-us/library/azure/dn931939.aspx. For an Azure Web App, the metric definitions should look similar to example screenshot below.

azure_web_app_metric_definitions_with_pointers

 

Once the available metric definitions are known, it is easy to retrieve the required metric values. Use the metric’s name ‘value’ (not the ‘localizedValue’) for any filtering requests (e.g. retrieve the ‘CpuTime’ and ‘Requests’ metric data points). The request / response information for this API call do not appear as an available task at https://msdn.microsoft.com/en-us/library/azure/dn931930.aspx. However, it is possible to do so, and the request URI is very similar to that of listing the metric definitions.

Method Request URI
GET https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/{resource-provider-namespace}/{resource-type}/{resource-name}/metrics?api-version=2014-04-01&$filter={filter}

For example, to retrieve just the Average Response Time and Requests metric data points for an Azure Web App for the 1 hour period from 2016-02-18 20:26:00 to 2016-02-18 21:26:00, with a granularity of 1 minute, the request URI would be as follows:

https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/collierwebapi2016/providers/Microsoft.Web/sites/collierwebapi2016/metrics?api-version=2014-04-01&$filter=%28name.value%20eq%20%27AverageResponseTime%27%20or%20name.value%20eq%20%27Requests%27%29%20and%20timeGrain%20eq%20duration%27PT1M%27%20and%20startTime%20eq%202016-02-18T20%3A26%3A00.0000000Z%20and%20endTime%20eq%202016-02-18T21%3A26%3A00.0000000Z

The result should be similar to that of the screenshot below.

azure_web_app_metric_values

 

Using the REST API is very helpful in terms of understand the available metric definitions, granularity, and related values. This information can be very helpful when using the Azure Insights Management Library.

Retrieving Metrics via the Insights Management Library

Just like working with the REST API, there are three basic steps for working with the Insights Management Library:

  1. Authenticate the Azure Insights request
  2. Retrieve the available metric definitions
  3. Retrieve the metric values

The first step is to authenticate by retrieving the JWT token from Azure AD. Assuming the Azure AD service principle is already configured, retrieving the token can be as simple as shown in the code sample below.

Get Auth Token
  1. private static string GetAccessToken()
  2. {
  3.     var authenticationContext = new AuthenticationContext(string.Format(“https://login.windows.net/{0}”, _tenantId));
  4.     var credential = new ClientCredential(clientId: _applicationId, clientSecret: _applicationPwd);
  5.     var result = authenticationContext.AcquireToken(resource: “https://management.core.windows.net/”, clientCredential: credential);
  6.     if (result == null)
  7.     {
  8.         throw new InvalidOperationException(“Failed to obtain the JWT token”);
  9.     }
  10.     string token = result.AccessToken;
  11.     return token;
  12. }

 

The primary class for working with the Insights API is the InsightsClient. This class exposes functionality to retrieve the available metric definitions and metric values, as seen in the sample code below:

Get Metric Data
  1. private static MetricListResponse GetResourceMetrics(TokenCloudCredentials credentials, string resourceUri, string filter, TimeSpan period, string duration)
  2. {
  3.     var dateTimeFormat = “yyy-MM-ddTHH:mmZ”;
  4.     string start = DateTime.UtcNow.Subtract(period).ToString(dateTimeFormat);
  5.     string end = DateTime.UtcNow.ToString(dateTimeFormat);
  6.     StringBuilder sb = new StringBuilder(filter);
  7.     if (!string.IsNullOrEmpty(filter))
  8.     {
  9.         sb.Append(” and “);
  10.     }
  11.     sb.AppendFormat(“startTime eq {0} and endTime eq {1}”, start, end);
  12.     sb.AppendFormat(” and timeGrain eq duration'{0}'”, duration);
  13.     using (var client = new InsightsClient(credentials))
  14.     {
  15.         return client.MetricOperations.GetMetrics(resourceUri, sb.ToString());
  16.     }
  17. }
  18. private static MetricDefinitionListResponse GetAvailableMetricDefinitions(TokenCloudCredentials credentials, string resourceUri)
  19. {
  20.     using (var client = new InsightsClient(credentials))
  21.     {
  22.         return client.MetricDefinitionOperations.GetMetricDefinitions(resourceUri, null);
  23.     }
  24. }

 

For the above code, the resource URI to use is the full path to the desired Azure resource. For example, to query against an Azure Web App, the resource URI would be:

/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.Web/sites/{site-name}/

 

It is also possible to query the metric data for a classic Azure Virtual Machine – just change the request URI to be appropriate for the classic VM:

/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/microsoft.classiccompute/virtualmachines/{vm-name}/

 

To find the resource URI for a desired resource, one approach is to use the https://resources.azure.com tool. Simply browse to the desired resource and then look at the URI shown, as in the screenshot below.

resource_explorer

 

For the full code sample, please see my GitHub repository available at https://github.com/mcollier/microsoft-azure-insights-demo/.

Thanks to Yossi Dahan’s blog post at https://yossidahan.wordpress.com/2015/02/13/reading-metric-data-from-azure-using-the-azure-insights-library/ for the inspiration.

azure_web_app_metric_value_listing

Deploy a WordPress Azure Web App with an Alternative MySQL Database

I was recently presented with an interesting question about Azure Web Apps, WordPress, and MySQL. While not necessarily a “hard” question, the answer wasn’t as readily available as I first anticipated. I thought I would share my experience here in hopes of helping others.

The Question

How can you deploy a WordPress site using Azure Web Apps that uses a MySQL database instance that is not a ClearDB database available in the Azure subscription?

Background

Normally when you create a WordPress site using Azure Web Apps you are presented with an option to select an existing ClearDB MySQL database, or create a new one. But what if you don’t want to use an existing instance or create a new one? What if you want to use a MySQL database instance deployed to an Azure VM or you have a ClearDB MySQL database that doesn’t show in the Azure Portal (e.g. one of the ClearDB Basic offerings)?

Create_WordPress_ClearDB_Normal

The Answer(s)

Like most technology related questions (or life in general), there are a few ways to solve this challenge. There is the “easy” way, and there is the more powerful, yet slightly more complicated, some would argue the “right” way.

The Easy Way

The easiest approach is to create a WordPress site with Azure Web Apps and select either an existing ClearDB database or create a new ClearDB database. Once the WordPress site is deployed, you can then change the database connection string in the wp-config.php file to be the database you want (e.g. a ClearDB Basic instance or a MySQL instance on an Azure VM).

  1. Let the WordPress site be deployed, but do not complete the installation. In other words, once the site is deployed, browsing to the site’s URL should result in the standard WordPress default installation prompt.
  2. WordPress_Default_Install_1
  3. Open the Kudu console by going to http://your-site-name.scm.azurewebsites.net. If you’re already signed into the Azure Portal, you should proceed through without any authentication challenge. Otherwise you’ll be challenged for your authentication credentials.
  4. Navigate to the Debug console (via the menu on the top). Browse to the \site\wwroot\ directory.
  5. Kudo_wpconfig_1
  6. Edit the wp-config.php file by clicking on the pencil icon to the left of the file name. Doing so will switch to an edit view for the file. Don’t click on the delete icon. . . that’d be a bad thing.
  7. Within the wp-config.php file, change the DB_NAME, DB_USER, DB_PASSWORD, and DB_HOST values to be that of the desired database. Save the file.
  8. WordPress_Default_Install_wpconfig_1
  9. Now reload your site – http://your-site-name.azurewebsites.net. This should load the default WordPress installation page prompting you to complete the WordPress installation.
  10. Complete the installation. This should use the database setting as configured in the wp-config.php file to finish the WordPress installation.
  11. If you created a free ClearDB database to start with, feel free to delete that ClearDB database.

The Alternative

And now the real fun begins! In this alternative approach, an Azure Resource Manager (ARM) template can be used to create the WordPress site on Azure Web Apps and wire up a database of your choosing. To make this happen you will need the ARM template and a MySQL database of your choosing.

To get the ARM template, my first thought was that I could download the template that the Azure Portal is using and simply modify the database connection details to be what I wanted. Wrong. The templates I tried turned out to be a bit more complicated that I wanted. However, they did provide a good start and helped me understand what I needed to do.

If you’re curious, you can get the templates by invoking the PowerShell script below.

# Retrieve all available items
$allGalleryItems = Invoke-WebRequest -Uri "https://gallery.azure.com/Microsoft.Gallery/GalleryItems?api-version=2015-04-01&includePreview=true" | ConvertFrom-Json

# Get all items published by WordPress
$allGalleryItems | Where-Object { $_.PublisherDisplayName -eq "WordPress" }
$allGalleryItems | Where-Object { $_.Identity -eq "WordPress.WordPress.1.0.0" }

# Save default template for all items under directory "C:\Templates"
$allGalleryItems | Foreach-Object {
$path = Join-Path -Path "C:\templates" -ChildPath $_.Identity
New-Item -type Directory -Path $path -Force

$.Artifacts | Where-Object { $.type -eq "template" } | ForEach-Object {
$templatePath = Join-Path -Path $path -ChildPath ( $_.Name + ".json" )

(Invoke-WebRequest -Uri $_.Uri).Content | Out-File -FilePath $templatePath
}
}

(original PowerShell sample from https://github.com/Azure/azure-powershell/issues/1064)

Using the ARM template obtained from the gallery sample as inspiration, I created a new ARM template. You can get the full sample on my GitHub repo at https://github.com/mcollier/AzureWebApp-WordPress-AlternativeDatabase.

"resources": [
{
"apiVersion": "2014-06-01",
"name": "[parameters('hostingPlanName')]",
"type": "Microsoft.Web/serverfarms",
"location": "[resourceGroup().location]",
"tags": {
"displayName": "HostingPlan"
},
"properties": {
"name": "[parameters('hostingPlanName')]",
"sku": "[parameters('sku')]",
"workerSize": "[parameters('workerSize')]",
"numberOfWorkers": 1
}
},
{
"apiVersion": "2014-06-01",
"name": "[variables('webSiteName')]",
"type": "Microsoft.Web/sites",
"location": "[resourceGroup().location]",
"tags": {
"[concat('hidden-related:', resourceGroup().id, '/providers/Microsoft.Web/serverfarms/', parameters('hostingPlanName'))]": "Resource",
"displayName": "Website"
},
"dependsOn": [
"[concat('Microsoft.Web/serverfarms/', parameters('hostingPlanName'))]"
],
"properties": {
"name": "[variables('webSiteName')]",
"serverFarm": "[parameters('hostingPlanName')]"
},
"resources": [
{
"apiVersion": "2014-11-01",
"name": "connectionstrings",
"type": "config",
"dependsOn": [
"[concat('Microsoft.Web/sites/', variables('webSiteName'))]"
],
"properties": {
"defaultConnection": {
"value": "[variables('dbConnectionString')]",
"type": 0
}
}
},
{
"apiVersion": "2014-06-01",
"name": "web",
"type": "config",
"dependsOn": [
"[concat('Microsoft.Web/sites/', variables('webSiteName'))]"
],
"properties": {
"phpVersion": "5.6"
}
},
{
"name": "MSDeploy",
"type": "extensions",
"location": "[resourceGroup().location]",
"apiVersion": "2014-06-01",
"dependsOn": [
"[concat('Microsoft.Web/sites/', variables('webSiteName'))]",
"[concat('Microsoft.Web/Sites/', variables('webSiteName'), '/config/web')]"
],
"tags": {
"displayName": "WordPressDeploy"
},
"properties": {
"packageUri": "https://auxmktplceprod.blob.core.windows.net/packages/wordpress-4.3.1-IIS.zip",
"dbType": "MySQL",
"connectionString": "[variables('dbConnectionString')]",
"setParameters": {
"AppPath": "[variables('webSiteName')]",
"DbServer": "[parameters('databaseServerName')]",
"DbName": "[parameters('databaseName')]",
"DbUsername": "[parameters('databaseUsername')]",
"DbPassword": "[parameters('databasePassword')]",
"DbAdminUsername": "[parameters('databaseUsername')]",
"DbAdminPassword": "[parameters('databasePassword')]"
}
}
}
]
}

The most relevant section is the MSDeploy resource extension (around line 60). It is this extension that deploys WordPress and gets the default database connection string set up. You provide the database server name, database name, database username and database password as input parameters to the ARM template. The ARM template will use those parameters to construct a database connection string in the proper format (set in a variable in the template).

Once the template is created, it can be deployed with a few lines of PowerShell:

#Login-AzureRmAccount

#NOTE - Ensure the correct Azure subscription is current before continuing. View all via Get-AzureRmSubscription -All
#Select-AzureRmSubscription -SubscriptionId "[your-id-goes-here]" -TenantId "[your-azure-ad-tenant-id-goes-here]"

$ResourceGroupName = "dg-wordpress-001"
$ResourceGroupLocation = "East US"
$TemplateFile = "azuredeploy.json"
$TemplateParametersFile = "azuredeploy.parameters.json"

Test-AzureRmResourceGroupDeployment -ResourceGroupName $ResourceGroupName `
-TemplateFile $TemplateFile `
-TemplateParameterFile $TemplateParametersFile `
-Verbose

# Create or update the resource group using the specified template file and template parameters file
New-AzureRmResourceGroup -Name $ResourceGroupName -Location $ResourceGroupLocation -Verbose -Force -ErrorAction Stop

New-AzureRmResourceGroupDeployment -Name ((Get-ChildItem $TemplateFile).BaseName + '-' + ((Get-Date).ToUniversalTime()).ToString('MMdd-HHmm')) `
-ResourceGroupName $ResourceGroupName `
-TemplateFile $TemplateFile `
-TemplateParameterFile $TemplateParametersFile `
-Force -Verbose

The reason I like this approach is that it is very clear what is being deployed. I can customize the template however I like, adding or removing additional resources as needed. Plus, I don’t have to go through that “create a database just to delete it” dance.

For instance, I can envision a version of this ARM template that may optionally set up a MySQL database on an Azure VM. Oh . . . look here, https://azure.microsoft.com/en-us/documentation/templates/wordpress-mysql-replication/. Someone already did mostly just that! That template could be modified to have some options to allow for the creation of a database in a few different configurations. Thanks for saving me some work. Naturally I found this after I went through all the work above. Go figure!  :)

Join me as a Cloud Solution Architect

Earlier this year I started a new chapter in my career as I joined Microsoft as a Cloud Solution Architect (CSA). Working for Microsoft has been a goal of mine for long time (since sometime in high school probably). Working for Microsoft in a role that allows me to continue working with Azure and solving interesting problems . . .  well, that’s just a lot of fun!

It is no secret that Microsoft is moving full steam ahead with Azure. The pace of innovation is mind blowing!  Part of my job as a CSA is to keep up on all the latest technologies in Azure – understanding how they really work and how to best leverage the various services/features to solve problems. It is a tough job, but someone has to do it. It is a fun job too!

The Azure platform is incredibly large. There are so many different services that is is in reality impossible to be an expert in all of the services. In that light, Microsoft has recently created a new role, the Data Solution Architect (DSA), that compliments the Cloud Solution Architect role. The DSA role focuses on data platform technologies such as Azure SQL Database, Machine Learning, HDInsight, etc. That allows the CSA role to focus on more of the compute and infrastructure services. Naturally, there is a bit of crossover in some areas . . . that is OK.

If you have ever wanted to work in Azure – in any capacity – now is a great time to join Microsoft. There are 292 current job openings currently showing on the Microsoft Careers site.

My team is currently hiring for both the Cloud Solution Architect and Data Solution Architect role. I currently work in SMS&P (that’s Microsoft speak for ‘Small and Midmarket Solutions & Partners’) in Central region for Microsoft. Current open positions include:

  • DSA – Austin, TX
  • CSA – Austin, TX
  • CSA – Minneapolis, MN
  • DSA – St. Louis, MO
  • DSA – Nashville, TN
  • CSA – Houston, TX
  • DSA – Chicago, IL
  • DSA – Milwaukee, WI
  • DSA – Indianapolis, IN

If you want to see the official job description for a CSA or DSA, check out the links below:

If a CSA or DSA sounds like something you would be interested, please contact me.

Being a CSA or DSA at Microsoft is a pretty cool job. But don’t just take my word for it; check out Walter Myer’s video below where he talks about his experience as a CSA.

Speaking at Visual Studio Live! New York

I’ll be speaking at Visual Studio Live!, September 28-October 1 in New York. Surrounded by your fellow industry professionals, Visual Studio Live! provides you with immediately usable training and education that will keep you relevant in the workforce.

I’ll be presenting the following sessions:
· Automating Your Azure Environment
· Inside the Azure Resource Manager

SPECIAL OFFER: As a speaker, I can extend $400 savings on the 4-day package. Register here: http://bit.ly/NYSPK07Reg

Amplify your knowledge at Visual Studio Live! New York — bring the issues that keep you up at night and prepare to leave this event with the answers, guidance and training you need. Register now: http://bit.ly/NYSPK07Reg