My New Azure Book

Last week Microsoft Press released the latest update to my book, Microsoft Azure Essentials: Fundamentals of Azure, Second Edition.   As the name indicates, this is an update to Microsoft Azure Essentials: Fundamental of Azure released in February 2015. I worked on both editions with my friend Robin Shahan.

The first edition of the book was relatively popular, with over 200,000 copies either downloaded or distributed in print. Thank you for the amazing support! I’m very happy so many people found the book to be helpful.fundamentalsofazure2e-thumbnail

When the Microsoft Press editor that worked on the first book initially approached me about working on an update, I was honestly a bit hesitant. Writing a technical book is a lot of work . . . especially a book on a platform such as Azure that seems to be constantly evolving. However, after many discussions with Robin, and my wife (getting spousal approval was very important as this effort would require many nights and weekends away from my family, which added a new baby to the party earlier this year), I decided to write the update. Thankfully Robin agreed to join in on the fun too. She’s a lot of fun to work with!

My thinking going into this process was there would be a few significant updates, but mostly it would be changing a few screenshots, updating a product name or two (i.e. Azure Web Sites became Azure Web Apps), and maybe including a bit of content around new services like Azure Service Fabric. It wouldn’t be that much work. Wrong!

Azure is a fast-moving platform. In the nearly 13 months since the first edition of the book was released to when I got started on the second edition, many things had changed. The second edition contains quite a few significant updates. The sections on Azure Web Apps, Azure Virtual Machines, Virtual Networks, Storage, and management tooling all received major edits. Key concepts such as Azure Resource Manager, use of the (new) Azure Portal, and the (almost entire) removal of Azure Cloud Services also received increased focused in the second edition. We also added a new chapter on Additional Azure Services which provides a brief introduction to several other Azure services Robin and I felt were important to understand. There are so many valuable services in the Azure platform, but we couldn’t discuss all of them in the book (or else we still might be working on the book, and it would be huge). These are services we personally felt were of significance for the broad majority of people. (Robin adds more info on the changes in her comment here.)

I would be remiss if I didn’t once again thank the Azure experts that volunteered their precious time to help review the book. Many of these people helped on the first edition too. Their expertise and honest feedback was critical in writing this book. Thank you!

I hope you find the second edition of the book to be a valuable asset to your journey to the public cloud with Microsoft Azure!

 

 

Retrieving Resource Metrics via the Azure Insights API

 

There are many options for configuring monitoring and alerting for Azure resources. The Azure portal will show some default metrics. It is also possible to enable more advanced or custom diagnostic settings. The links below provide additional details on enabling diagnostics and monitoring for two popular Azure compute resources, Azure Web Apps and Azure Virtual Machines. Visual Studio Application Insights can also be leveraged to monitor the usage and performance of applications.

As previously mentioned, the Azure portal will show some default metrics for many Azure resources. For example, the screenshot below shows the monitoring tile for an Azure Web App, which has been configured to display three key values: Average Response Time, CPU Time, and Requests.

azure_web_app_default_metric_tile

 

The metric values displayed on this tile are not retained indefinitely. For an Azure Web App, the retention policy is as follows:

  • Minute granularity – 24 hour retention
  • Hour granularity – 7 day retention
  • Daily granularity – 30 day retention

By using the Azure Insights API it is possible to programmatically retrieve the available default metric definitions (the type of metric such as CPU Time, Requests, etc.), granularity, and metric values. With the ability to programmatically retrieve the data, comes the ability to save the data in a data store of your choosing. For example, that data could be persisted to Azure SQL Database, DocumentDB, or Azure Data Lake. From there you could perform whatever additional analysis is desired.

It should be noted that the Azure Insights API is not the same as Application Insights.

Besides working with various metric data points, the Insights API allows you to manage things like alerts, autoscale settings, usage quotas, and more. Check out the full list via the Azure Insights REST API Reference documentation.

The remainder of this post will discuss using the Insights API to learn more about the default metrics available for Azure resources.

Investigating Available Metrics via the Insights REST API

There are three basic steps for working with the Insights REST API:

  1. Authenticate the Azure Insights request
  2. Retrieve the available metric definitions
  3. Retrieve the metric values

The first step is to authenticate the Azure Insights API request. As the Azure Insights API is an Azure Resource Manager based API, it requires authentication via Azure Active Directory (Azure AD). The easiest way (in my opinion at least) to set up authentication is by creating an Azure AD service principal and retrieve the authentication (JWT) token. The sample script below demonstrates creating an Azure AD service principle via PowerShell. For a more detailed walkthrough, please reference the guidance at https://azure.microsoft.com/en-us/documentation/articles/resource-group-authenticate-service-principal/#authenticate-service-principal-with-password—powershell. It is also possible to create a service principle via the Azure portal.

Create a Service Principle
  1. # Instructions at https://azure.microsoft.com/en-us/documentation/articles/resource-group-authenticate-service-principal/
  2. $pwd = “[your-service-principle-password]”
  3. $subscriptionId = “[your-azure-subscription-id]”
  4. Login-AzureRmAccount
  5. Select-AzureRmSubscription -SubscriptionId $subscriptionId
  6. $azureAdApplication = New-AzureRmADApplication `
  7.                         -DisplayName “Collier Web Metrics Demo” `
  8.                         -HomePage https://localhost/webmetricdemo” `
  9.                         -IdentifierUris https://localhost/webmetricdemo” `
  10.                         -Password $pwd
  11. New-AzureRmADServicePrincipal -ApplicationId $azureAdApplication.ApplicationId
  12. New-AzureRmRoleAssignment -RoleDefinitionName Reader -ServicePrincipalName $azureAdApplication.ApplicationId
  13. $subscription = Get-AzureRmSubscription -SubscriptionId $subscriptionId
  14. $creds = Get-Credential -UserName $azureAdApplication.ApplicationId -Message “Please use your service principle credentials”
  15. Login-AzureRmAccount -Credential $creds -ServicePrincipal -TenantId $subscription.TenantId

Once the authentication setup step is complete, it is possible to execute queries against the Azure Insights REST API. There are two helpful queries:

  1. List the metric definitions for a resource
  2. Retrieve the metric values

Details on listing the metric definitions for a resource is documented at https://msdn.microsoft.com/en-us/library/azure/dn931939.aspx. For an Azure Web App, the metric definitions should look similar to example screenshot below.

azure_web_app_metric_definitions_with_pointers

 

Once the available metric definitions are known, it is easy to retrieve the required metric values. Use the metric’s name ‘value’ (not the ‘localizedValue’) for any filtering requests (e.g. retrieve the ‘CpuTime’ and ‘Requests’ metric data points). The request / response information for this API call do not appear as an available task at https://msdn.microsoft.com/en-us/library/azure/dn931930.aspx. However, it is possible to do so, and the request URI is very similar to that of listing the metric definitions.

Method Request URI
GET https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/{resource-provider-namespace}/{resource-type}/{resource-name}/metrics?api-version=2014-04-01&$filter={filter}

For example, to retrieve just the Average Response Time and Requests metric data points for an Azure Web App for the 1 hour period from 2016-02-18 20:26:00 to 2016-02-18 21:26:00, with a granularity of 1 minute, the request URI would be as follows:

https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/collierwebapi2016/providers/Microsoft.Web/sites/collierwebapi2016/metrics?api-version=2014-04-01&$filter=%28name.value%20eq%20%27AverageResponseTime%27%20or%20name.value%20eq%20%27Requests%27%29%20and%20timeGrain%20eq%20duration%27PT1M%27%20and%20startTime%20eq%202016-02-18T20%3A26%3A00.0000000Z%20and%20endTime%20eq%202016-02-18T21%3A26%3A00.0000000Z

The result should be similar to that of the screenshot below.

azure_web_app_metric_values

 

Using the REST API is very helpful in terms of understand the available metric definitions, granularity, and related values. This information can be very helpful when using the Azure Insights Management Library.

Retrieving Metrics via the Insights Management Library

Just like working with the REST API, there are three basic steps for working with the Insights Management Library:

  1. Authenticate the Azure Insights request
  2. Retrieve the available metric definitions
  3. Retrieve the metric values

The first step is to authenticate by retrieving the JWT token from Azure AD. Assuming the Azure AD service principle is already configured, retrieving the token can be as simple as shown in the code sample below.

Get Auth Token
  1. private static string GetAccessToken()
  2. {
  3.     var authenticationContext = new AuthenticationContext(string.Format(“https://login.windows.net/{0}”, _tenantId));
  4.     var credential = new ClientCredential(clientId: _applicationId, clientSecret: _applicationPwd);
  5.     var result = authenticationContext.AcquireToken(resource: “https://management.core.windows.net/”, clientCredential: credential);
  6.     if (result == null)
  7.     {
  8.         throw new InvalidOperationException(“Failed to obtain the JWT token”);
  9.     }
  10.     string token = result.AccessToken;
  11.     return token;
  12. }

 

The primary class for working with the Insights API is the InsightsClient. This class exposes functionality to retrieve the available metric definitions and metric values, as seen in the sample code below:

Get Metric Data
  1. private static MetricListResponse GetResourceMetrics(TokenCloudCredentials credentials, string resourceUri, string filter, TimeSpan period, string duration)
  2. {
  3.     var dateTimeFormat = “yyy-MM-ddTHH:mmZ”;
  4.     string start = DateTime.UtcNow.Subtract(period).ToString(dateTimeFormat);
  5.     string end = DateTime.UtcNow.ToString(dateTimeFormat);
  6.     StringBuilder sb = new StringBuilder(filter);
  7.     if (!string.IsNullOrEmpty(filter))
  8.     {
  9.         sb.Append(” and “);
  10.     }
  11.     sb.AppendFormat(“startTime eq {0} and endTime eq {1}”, start, end);
  12.     sb.AppendFormat(” and timeGrain eq duration'{0}'”, duration);
  13.     using (var client = new InsightsClient(credentials))
  14.     {
  15.         return client.MetricOperations.GetMetrics(resourceUri, sb.ToString());
  16.     }
  17. }
  18. private static MetricDefinitionListResponse GetAvailableMetricDefinitions(TokenCloudCredentials credentials, string resourceUri)
  19. {
  20.     using (var client = new InsightsClient(credentials))
  21.     {
  22.         return client.MetricDefinitionOperations.GetMetricDefinitions(resourceUri, null);
  23.     }
  24. }

 

For the above code, the resource URI to use is the full path to the desired Azure resource. For example, to query against an Azure Web App, the resource URI would be:

/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.Web/sites/{site-name}/

 

It is also possible to query the metric data for a classic Azure Virtual Machine – just change the request URI to be appropriate for the classic VM:

/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/microsoft.classiccompute/virtualmachines/{vm-name}/

 

To find the resource URI for a desired resource, one approach is to use the https://resources.azure.com tool. Simply browse to the desired resource and then look at the URI shown, as in the screenshot below.

resource_explorer

 

For the full code sample, please see my GitHub repository available at https://github.com/mcollier/microsoft-azure-insights-demo/.

Thanks to Yossi Dahan’s blog post at https://yossidahan.wordpress.com/2015/02/13/reading-metric-data-from-azure-using-the-azure-insights-library/ for the inspiration.

azure_web_app_metric_value_listing

Speaking at Visual Studio Live! New York

I’ll be speaking at Visual Studio Live!, September 28-October 1 in New York. Surrounded by your fellow industry professionals, Visual Studio Live! provides you with immediately usable training and education that will keep you relevant in the workforce.

I’ll be presenting the following sessions:
· Automating Your Azure Environment
· Inside the Azure Resource Manager

SPECIAL OFFER: As a speaker, I can extend $400 savings on the 4-day package. Register here: http://bit.ly/NYSPK07Reg

Amplify your knowledge at Visual Studio Live! New York — bring the issues that keep you up at night and prepare to leave this event with the answers, guidance and training you need. Register now: http://bit.ly/NYSPK07Reg

New Book – Microsoft Azure Essentials

In early February, Microsoft Press released a free new eBook entitled Fundamentals of Azure: Microsoft Azure Essentials. This is a book I, along with Azure MVP Robin Shahan spent several months on in late 2014. To see if finally be published is truly exciting!Book Cover

The goal of the book is to serve as primer for those new to cloud computing in general, and the Microsoft Azure platform specifically. We do not go too deep on the topics, and we don’t cover all the services available in Azure. We cover the core concepts people need to get started and be successful.

Writing a book is hard work – much harder than I originally anticipated. I had friends tell me it would be a lot of work – they were right! Writing a book on Azure adds to the challenge, I think. The reason is Azure is a constantly evolving platform. It’s hard to write a book on something that changes several times a month. We tried to keep up with the changes as best we could. But, inevitably some things will be inaccurate by the time to book is published or shortly thereafter.

Writing a book is also fun! While many nights and weekends were sacrificed to write this book, it was a very rewarding experience. I know I learned a lot while writing this book. I had many late night Skype calls with Robin to talk through how we wanted topics to flow. I had many great email conversations with the Azure MVPs that helped to provide early feedback on the book. A lot of work, but fun – and that’s the best kind of work!

You can find the book at http://aka.ms/fundamentalsofazure and get it in PDF, EPUB, or Mobi formats.

Topics covered in the book include.

  • Getting started with Azure: Understand what cloud computing is, visit the management
    portals, and learn about billing.
  • Websites and Cloud Services: Learn about Azure Websites, from deployment to monitoring, and gain an understanding of the web and worker roles used in Azure Cloud Services.
  • Virtual Machines: Explore the basic features of Azure Virtual Machines, including how to create, configure, and manage them.
  • Storage: Read about the basics of Azure Storage, including blobs, tables, queues, and file shares.
  • Virtual Networks: Learn the basics of virtual networks, including how to create one, and why a virtual network might be necessary. This also covers site-to-site and point-to-site networking, as well as ExpressRoute.
  • Databases: Explore two relational database options available in Azure: Azure SQL Database and SQL Server in Azure Virtual Machines.
  • Azure Active Directory: Explore basic features of Azure AD, including creating a directory, users and groups, and using the application gallery.
  • Management Tools: Explore three common tools for working with Azure: Visual Studio 2013 and the Azure SDK, Azure PowerShell cmdlets, and the Cross-Platform Command-Line Interface
  • Business Scenarios: Explore four common scenarios for utilizing Azure features: development and test, hybrid, application and infrastructure modernization, and Azure Mobile Services.

Determine the RDP User for Web/Worker Role

Note: This post is cross-post with http://blogs.msdn.com/b/cloud_solution_architect.

It is widely known that it is possible to add Remote Desktop (RDP) to a Cloud Service (web/worker role). Adding RDP is often done as a support mechanism – enabling the desire to see what is happening with the virtual machine and how the deployed code is working, or not.

Options to Enable Remote Desktop

There are two ways to add Remote Desktop support to a Cloud Service deployment:

1. Using Visual Studio, Remote Desktop can be added when the service is deployed by clicking the “Enable Remote Desktop for all roles” option in the publishing wizard. This approach will add two modules, RemoteAccess and RemoteForwarder, into the service model. You can read more on MSDN about setting up the connection in the service model.

 VisualStudioPublishWizard

2. Using the Azure Management Portal, for the desired Cloud Service, first navigate to the CONFIGURE section and then select the REMOTE option in the bottom command bar. You can read more about this approach at http://azure.microsoft.com/en-us/documentation/articles/cloud-services-how-to-configure/#remoteaccess.

 AzureManagementPortal-EnableRDP

Determine the Username

There are scenarios in which Remote Desktop is disabled when the Cloud Service is deployed, and only enabled in the event of a support incident when it is deemed RDP is the best support option. In this case, a co-administrator can enable RDP via the Azure Management Portal (as mentioned previously).  When first enabling RDP you set the username and password for the local machine account to be created on the role instance.  But what about after RDP is enabled – how do you know the local username that was created?  Is this information logged?

Enabling Remote Desktop in a Cloud Service is done via an extension. To learn more about extensions with Cloud Services and using them to enable Remote Desktop, I would encourage you to read Azure MVP Gaurav Mantri’s excellent blog post on the topic. Gaurav’s post shows how to add the Remote Desktop extension via the Service Management API. This information can be also useful when needing to retrieve details about a previously added Remote Desktop extension.

When Remote Desktop is enabled by using the extension model (e.g. enabling Remote Desktop via the portal), the operation logs in the Azure Management Portal (in the Management Services section) will show a new operation – AddHostedServiceExtension. If you were to look at the details for this extension, you would see something similar to the following:

<SubscriptionOperation xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
     <OperationId>ae0fcdb1-xxxxxxxxxxxxxx</OperationId>
     <OperationObjectId>/0bbbc191-xxxxxxxxxxxxxxxxx/services/hostedservices/rdpdemo01</OperationObjectId>
     <OperationName>AddHostedServiceExtension</OperationName>
     <OperationParameters xmlns:d2p1="http://schemas.datacontract.org/2004/07/Microsoft.WindowsAzure.ServiceManagement">
          <OperationParameter>
               <d2p1:Name>subscriptionID</d2p1:Name>
               <d2p1:Value>0bbbc191-xxxxxxxxxxxxxxxxx</d2p1:Value>
          </OperationParameter>
          <OperationParameter>
               <d2p1:Name>serviceName</d2p1:Name>
               <d2p1:Value>rdpdemo1</d2p1:Value>
          </OperationParameter>
          <OperationParameter>
               <d2p1:Name>input</d2p1:Name>
               <d2p1:Value><?xml version="1.0" encoding="utf-16"?><Extension xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/windowsazure"><ProviderNameSpace>Microsoft.Windows.Azure.Extensions</ProviderNameSpace><Type>RDP</Type><Id>RDP-a2d56066-1a78-40a9-98f3-e0a941a55a59</Id><Version>1.*</Version></Extension></d2p1:Value>
          </OperationParameter>
     </OperationParameters>
     <OperationCaller>
          <UsedServiceManagementApi>true</UsedServiceManagementApi>
          <UserEmailAddress>xxxxxxxxxx</UserEmailAddress>
          <SubscriptionCertificateThumbprint />
          <ClientIP>70.xx.xx.xx</ClientIP>
     </OperationCaller>
     <OperationStatus>
          <ID>ae0fcdb1-xxxxxxxxxxx</ID>
          <Status>Succeeded</Status>
          <HttpStatusCode>200</HttpStatusCode>
     </OperationStatus>
     <OperationStartedTime>2015-01-23T20:55:48Z</OperationStartedTime>
     <OperationCompletedTime>2015-01-23T20:55:55Z</OperationCompletedTime>
     <OperationKind>AddHostedServiceExtensionOperation</OperationKind>
</SubscriptionOperation>

A key piece of information in the above XML is the “input” OperationParameter. There are two key pieces of data here – the Type of the extension is “RDP” and the ID.  The type of “RDP” lets us know that we are looking at the correct extension – the extension which added Remote Desktop support to the Cloud Service. The ID provides a unique identifier for the specific extension’s configuration information. With that ID, we can use the Get Extension operation in the Azure Service Management  API to retrieve public configuration, which will then provide the username created when enabling Remote Desktop.

In order to retrieve the public configuration, there are three steps you will need to perform:

  1. Set up authentication with the Azure Service Management API. See the Authenticating Service Management Requests topic in MSDN for more information.
  2. Make a request to the Get Extension operation, providing the RDP extension ID retrieved from the Azure Management Portal.  For example, https://management.core.windows.net/0bbbc191-xxxxxxxxxxxxxxxx/services/hostedservices/rdpdemo1/extensions/RDP-a2d56066-1a78-40a9-98f3-e0a941a55a59. This can be done using a tool such as Fiddler, or by writing code against the Service Management API. The response should contain the base64 encoded public configuration data.
  3. Decode the public configuration. Once decoded, you will be able to find the username used when enabling RDP.
<PublicConfig xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
     <UserName>mcollier</UserName>
     <Expiration>2015-02-04 05:00:00Z</Expiration>
</PublicConfig>

Speaking at VS Live! Las Vegas

I’ll be speaking at Visual Studio Live!, March 16-20 in Las Vegas. Surrounded by your fellow industry professionals, Visual Studio Live! provides you with immediately usable training and education that will keep you relevant in the workforce.

I’ll be presenting the following sessions:

SPECIAL OFFER: As a speaker, I can extend $500 savings on the 5-day package. Register here: http://bit.ly/LVSPK09_Reg

Make sure to sure to use the code ‘LVSPK09’ to get the discount!

 

Amplify your knowledge at Visual Studio Live! Las Vegas — bring the issues that keep you up at night and prepare to leave this event with the answers, guidance and training you need.  Register now: http://bit.ly/LVSPK09_Reg

Azure Worker Role Changes in SDK 2.4

It’s been quite a while since we’ve seen any significant (ok, really any) changes in the boilerplate template code Visual Studio generates for Azure Cloud Services (web or worker roles). For as long as I can remember, the code has looked like this:

public override void Run()
{
    // This is a sample worker implementation. Replace with your logic.
    Trace.TraceInformation("WorkerRole.23 entry point called");

    while (true)
    {
       Thread.Sleep(10000);
       Trace.TraceInformation("Working");
    }
}

Pretty basic, right?  Just do some work forever.  If there is a failure (i.e. an unhandled exception), it’ll bubble up and out of the Run method, effectively crashing the role and Azure will restart it. The problem with this is that it didn’t effectively handle the case when the role stops – when OnStop is called. If your code was doing something at the that time, too bad, so sad.  You’re done. Microsoft released some guidance way back in January 2013 on The Right Way to Handle Azure OnStop Events. Unless you knew of this blog post, or were really good at your Bing-fu, it was too easy to just use the Visual Studio generated code. And then something happens, your code doesn’t do what you think it should (even though it does exactly what it is told to do), and bad words are uttered silently, and feelings get hurt.

Staring with Azure SDK 2.4, there is new boilerplate code generated by Visual Studio.  Check this out:

public class WorkerRole : RoleEntryPoint
{
    private readonly CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
    private readonly ManualResetEvent runCompleteEvent = new ManualResetEvent(false);

    public override void Run()
    {
        Trace.TraceInformation("WorkerRole.24 is running");

        try
        {
            this.RunAsync(this.cancellationTokenSource.Token).Wait();
        }
        finally
        {
            this.runCompleteEvent.Set();
        }
    }

    public override void OnStop()
    {
        Trace.TraceInformation("WorkerRole.24 is stopping");

        this.cancellationTokenSource.Cancel();
        this.runCompleteEvent.WaitOne();

        base.OnStop();

        Trace.TraceInformation("WorkerRole.24 has stopped");
    }

    private async Task RunAsync(CancellationToken cancellationToken)
    {
       // TODO: Replace the following with your own logic.
       while (!cancellationToken.IsCancellationRequested)
       {
           Trace.TraceInformation("Working");
           await Task.Delay(7000);
       }
    }
}

At first blush, this looks more complicated. But it really isn’t that bad. In a way, it’s doing what that Microsoft blog post from 2013 was indicating, just using cancellation tokens.

The OnStop method is actually overridden in the template now. There, the cancellation token is triggered to indicate the running thread should be cancelled, and then wait for that to happen. In the RunAsync method, we’re just waiting for that cancellation request, otherwise continue doing some business. Easy.

Keep in mind that there is still the 5 minute limit to finish, or else Azure will terminate the process.