Month: March 2018

Using the Face API from Microsoft Cognitive Services (part 2)–Face Verification

Callon Campbell [MVP] on March 31, 2018

In part 1 of this series I showed you how to create a Face API subscription from Microsoft Cognitive Services and then use the Face API library to detect faces from an image. In this post we’ll expand on the previous to include Face Verification. Let’s get started.

Picking up where we left off, we will want to detect the most prominent face from an image and then use the detected face and verify it to see if they belong to the same person.

1. I refactored the code in the BrowsePhoto method to return an image that was selected. This method is then used by both the Identification and Verification images processes.

2. I refactored the UI to show 2 different images files, so means there is now 2 click events to identify the person in the image and then use this identification to verify its the same person when we load up another image. Both of these events can be seen here:

3. Finally we will be using the Face API VerifyAsync method to check to faces and determine if they belong to the same person.

4. Now let’s run the application across a few images and see how well it performs with two images of me from different years. In the first result I have an image from 10+ years ago and the Face API has come back that its 66% certain it’s the same person.

How about using something more recent. In this next test run the Face API again detects its 75% certain its the same person.

Wrap up

As you can see I’m able to use the Face API from Microsoft Cognitive Services to not only detect by also verify identity. The Face API provides other methods that can be used for grouping, people together and training it to recognize specific people with their identification method.The Face API has also recently been updated to support a large group of people (1,000 in the free tier and over 1,000,000 in the paid tier).

Enjoy!

References

Sample Code

Face API Documentation

AI Azure Development

Using the Face API from Microsoft Cognitive Services (part 1)–Face Detection

Callon Campbell [MVP] on March 31, 2018

Earlier this month I wrote about giving your applications a more human side with Microsoft Cognitive Services, which provides a number of API’s that you can start using immediately in your applications. Today I’ll dive into the vision API’s and show you how you can leverage the Face API to detect faces in your images.

What is the Face API?

The Face API provides facial and emotion recognition and location in an image. There are 5 main areas for this API:

– Face detection
– Face verification
– Find similar faces
– Face grouping
– Face identification

Potential uses for this technology include facial login, photo tagging, and home monitoring. You can also use it for attribute detection to know age, gender, facial hair, whether the person is wearing a hat, wearing glasses, or has a beard. This API can also be used to determine if two faces belong to the same person, identify previously tagged people, find similar-looking faces in a collection.

So let’s get started with creating an Face API resource and then a small application to detect faces. In the next post I’ll extend this example to do face verification to determine if it’s the same person.

Step 1 – Requirements

To get started with Microsoft Cognitive Services and specifically the Face API you will need to have an Azure Subscription. If you don’t have one you can get a free trial subscription which includes $250 of credits to be used for any Azure services.

You will also need to have Visual Studio 2017 installed, which you can download for free.

Step 2 – Subscribe to the Face API

1. Log in to the Azure portal and click on the Create a resource link in top left corner. From here select AI + Cognitive Services and then select Face API as shown here:

2. Give your Face API a name, select your subscription, location, resource group and then select the F0 Free tie for pricingr:

3. After a few seconds your Face API subscription will be created and ready for you to start using. At this point you will need to get two items, your subscription key and your endpoint location.

The endpoint URL is shown on the Overview section and your subscription keys are located under Keys in the Resource Management section as shown here:

Now that we have the subscription key and endpoint let’s create our application.

Step 3 – Create new Application and reference the Face API

1. Open Visual Studio and from the File menu, click on New then Project. From here you can select any type of application but for me I’m going to create a new WPF application in C#. This code will also work with Xamarin.Forms project if you wanted to try this out for mobile.

2. Go to the Solution Explorer pane in Visual Studio, right click your project and then click Manage NuGet Packages.

3. Click on the Include prerelease checkbox and then search for Microsoft.Azure.CognitiveServices.Vision.Face. You might be wondering why are these API’s still in preview? Well the Cognitive Services API’s were previously called Microsoft.ProjectOxford.* and are now being moved over to Microsoft.Azure.CognitiveServices.*. Once that migration is complete they should be out of prerelease and is what you should be using from then on.

4.Now let’s go to the code and configure the Face API client library.

Step 4 – Configure the Face API Client Library

1. Open up your MainWindow.cs file and declare a new FaceServiceClient instance as shown here

2. Insert your Face API subscription key and endpoint. Replace “YOUR-SUBSCRIPTION-KEY-GOES-HERE” with your subscription key from step 2. Do the same for the second parameter which is your endpoint URL.

Step 5 – Upload images, detect faces, and show facial attributes

I wont walk through the entire code as you can do that on my GitHub repository. Instead in this step I’ll show you how I used the Face API to detect the faces, draw a square around each detected face, and finally show you the facial attributes when the mouse hovers over a detected face.

It’s worth mentioning that the maximum size of the image to upload is 4 MB.

As highlighted above you will take a photo you have and upload it to the Face API where it will detect an array of faces. The largest face in the image is usually what is returned first in the array. Using the DetectAsync method, you have the option to pass in an IEnumerable of FaceAttributeTypes. Just declare a list of the attributes you want back in the results like so:

The second highlighted code shows were we store the facial attributes returned for each face. The GetFaceDescription method is used when you mouse over a detected face and you want to show the attributes that were returned from the Face API:

Now let’s run our application and try detecting some faces for an image containing one or more faces. After a few seconds the API will return back with the results. As you can see we’re drawing blue squares for the makes and pink for the females, and when you hover your mouse over one of the faces I’m displaying the description of all the facial attributes returned by the API.

Wrap up

As you can see its very easy to add AI to your application with Microsoft Cognitive Services. Today I showed you how you can leverage the Face API for facial recognition.

Enjoy!

Resources

Sample Code

Face API Documentation

Development

Updates to the New Project Dialog in Visual Studio 2017

Callon Campbell [MVP] on March 28, 2018

With the release of Visual Studio 2017 Update 6 (version 15.6.x), you might have noticed that there was an update done to the New Project dialog to move the .NET Framework version selector down to the bottom below the solution name. This makes the New Project dialog cleaner and shows you which version of the .NET Framework will be used when creating your project.

The Framework selector disappears when selecting non .NET Framework projects like ASP.NET Core, UWP, etc.

Enjoy!

Azure

How to Lock Azure Resources and Prevent Unexpected Changes or Deletions

Callon Campbell [MVP] on March 24, 2018

Management locks can help you prevent accidental deletion or modification of your Azure resources. You can manage these locks from one of the following…the Azure Portal, ARM Templates, PowerShell, Azure CLI, or the REST API. To view, add, or delete locks, go to the Locks section of any resource’s settings blade. In the Azure Portal, the locks are called Delete and Read-Only respectively.

There are two possible types of locks on a resource:

CanNotDetele – This means authorized users can still read and modify a resource, but they can’t delete the resource.
ReadOnly – This means authorized users can read a resource, but they can’t delete or update the resource. Applying this lock is similar to restricting all authorized users to the permissions granted by the Reader role.

When a lock is applied at the parent level, all resources within that scope inherit the same lock. This applies to any resources you add later on to this parent resource. Resource locks do not restrict how a particular resource functions and only resource changes are restricted, but the most restrictive lock will always take precedence.

Creating a Lock using the Portal

1. In the portal, go to the particular resource you want to lock. In this case it’s a Resource Group but it could be any Resource, a Resource Group, or a Subscription and then click on the Lock option under the Settings section:

2. To add a lock click on the Add button:

3. Give your lock a name and the type of lock (Delete or Read-Only) and then click on the OK button:

Your resources are now locked. If you try to delete a resource that is locked you will see the following warning which prevents you from deleting the particular resource:

Unlocking a Resource

To unlock the resource click on the ellipse (…) button and click on the Delete option:

Using resource locks is a must and really prevents an “oops…I deleted the wrong resource” situation which leads to accidental and hard to recover from downtown.

Enjoy!

Resources

https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-lock-resources

Lock Down your Azure Resources

Remove Locks from Azure Resources

Development

Getting Started with Application Insights for ASP.NET Core

Callon Campbell [MVP] on March 24, 2018

In my previous posts I gave a quick Introduction to Application Insights and then I showed you how to Disable Application Insights from your app. In this post I’ll walk you through creating an ASP.NET Core application and then configuring it with Application Insights. Let’s get started.

Configuring your app for Application Insights

Start by creating a new ASP.NET Core application (this also applied to non-core ASP.NET applications). Once the application is created right click on the project file in the context menu look for Configure Application Insights… and then click on it.

You will see that the SDK has already been added to your application. Next click on the Start Free button to start using Application Insights.

You will need to have an existing Azure Subscription. If you don’t already have one you can create one for free and start with a $250 credit for 30 days + you will have access to popular services for 12 months + there are over 25 services that are always free. Now that you have your Azure Subscription, login with your Microsoft Account, select your Subscription and then a Resource. These can always be easily changed later on if need be.

You will now have access to the free plan which comes with 1 GB / Month of data included and data retention is 90 days. Click on he Register button to finish the configuration:

Now that Application Insights is configured for your application you have access to a wealth of information with the click of a button.

Accessing the Application Insights Telemetry from Visual Studio

You can search your Application Insights results from either the Azure Portal or from within Visual Studio. To use Visual Studio go to the View menu, select Other Windows and then Application Insight Search. You will then get view of the telemetry for the last 24 hours as shown below from a sample API I have. From here you can filter the telemetry and dive down into specific events.

Another nice feature is that Application Insights telemetry data including any exceptions that have been captured will show up in the CodeLens information as shown here:

There is a lot of value from using Application Insights in any of your applications. I hope you take a look and try it out for yourself.

Enjoy!

References

https://azure.microsoft.com/en-us/services/application-insights/

https://docs.microsoft.com/en-us/azure/application-insights/app-insights-asp-net-core

Development

How to Disable Azure Application Insights in ASP.NET Core

Callon Campbell [MVP] on March 23, 2018

In my previous post I showed you how easy it was to get started with an Introduction to Application Insights for your ASP.NET Core application. However what if you you don’t want Application Insights? You might notice in your Output pane when running your app that it’s still partially enabled for you out of the box. I’ll walk you through what I mean by it being partially enabled and then how you can go about hiding it until such time you decide to fully turn it on. Let’s get started.

Start off by creating a new ASP.NET Core application (see below) and then immediately run it.

You will then notice that you will see the following statements in your Output pane:

Application Insights Telemetry (unconfigured): {“name”:”Microsoft.ApplicationInsights.Dev.Message”,”time”:”2018-03-24T03:39:26.5327026Z”,”tags”:{“ai.application.ver”:”1.0.0.0″,”ai.operation.parentId”:”|80d77757-4707b4b80d71a9b3.”,”ai.internal.sdkVersion”:”aspnet5c:2.1.1″,”ai.operation.id”:”80d77757-4707b4b80d71a9b3″,”ai.internal.nodeName”:”LT2206″,”ai.location.ip”:”127.0.0.1″,”ai.cloud.roleInstance”:”LT2206″,”ai.operation.name”:”GET Values/Get”,”ai.user.id”:”6RWa2″},”data”:{“baseType”:”MessageData”,”baseData”:{“ver”:2,”message”:”Executed action WebApplication5.Controllers.ValuesController.Get (WebApplication5) in 205.1085ms”,”severityLevel”:”Information”,”properties”:{“DeveloperMode”:”true”,”{OriginalFormat}”:”Executed action {ActionName} in {ElapsedMilliseconds}ms”,”ActionName”:”WebApplication5.Controllers.ValuesController.Get (WebApplication5)”,”AspNetCoreEnvironment”:”Development”,”ElapsedMilliseconds”:”205.1085″,”CategoryName”:”Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker”}}}}

You might be wondering why is it doing this and how can I disable it?

The easiest way to disable Application Insights without going through the process of ripping it out is to just disable it. You can do this by accessing TelemetryConfiguration.Active.DisableTelemetry and setting this to true. What I would recommend doing is to add a static method to your Startup.cs file and call this method from your Configure method like so:

Now when you run your application and look in the Output pane you will no longer see any statement pertaining to Application Insights.

I see a great deal of value of keeping Application Insights and using it in all your applications, so if you need to disable it then maybe do this when running in debug mode by using a conditional attribute on the method.

Enjoy!

References

https://github.com/aspnet/Home/issues/2051

AI Azure

Give your solutions a more human side with Microsoft Cognitive Services

Callon Campbell [MVP] on March 13, 2018

Making AI Possible

Today there are three mega trends converging that are making AI possible:

1. Big Compute
2. Powerful Algorithms
3. Massive Data

Microsoft is in a unique position to help you take advantage of these trends with three important assets:

1. Microsoft Azure, providing the best cloud for developers
2. Breakthrough in AI Innovations, through Microsoft Azure and their AI resources this innovation is brought to you as a developer
3. Data. Microsoft Graph gives you access to the most important data for your business and/or application, your data!

Microsoft has a strong vision that AI should be democratized and be available to everyone – developers, data scientists, enterprises, and yes even your dog. Microsoft has been involved and conducting research into AI for the last couple decades and infusing it into their products and services (Bing, Xbox, Office 365, Skype, Cortana, LinkedIn, etc). This research eventually found its way into a product known as Microsoft Cognitive Services.

Introducing Microsoft Cognitive Services

Microsoft Cognitive Services, formerly known as “Project Oxford” was first announced at Build 2016 conference and released as a preview. This is a rich collection of cloud-hosted APIs that let’s developers add AI capabilities such as vision, speech, language, knowledge and search into any application across any platform (Windows, Mac, iOS, Android, and Web) using simple RESTful APIs and/or SDKs (NuGet packages). Rather than having to deal with the complexities that come with machine learning, Cognitive Services provides simple APIs that handle common use cases, such as recognizing speech or performing facial recognition on an image.These APIs are based off machine learning and fit perfectly into the conversation-as-a-platform philosophy.

With Microsoft Cognitive Services, you can give your applications a human side. To date there are currently 29 APIs across 5 categories of Vision, Speech, Language, Knowledge, and Search. Let’s take a look at each of these categories:

Vision – From faces to feelings, allow your apps to understand images and videos

Speech – Hear and speak to your users by filtering noise, identifying speakers, and understanding intent

Language – Process text and learn how to recognize what users want

Knowledge – Tap into rich knowledge amassed from the web, academia, or your own data

Search – Access billions of web pages, images, videos and news with the power of Bing API’s

Labs – Microsoft Cognitive Services Labs is an early look at emerging technologies that you can discover, try and provide feedback before they become generally available

Why Use Microsoft Cognitive Services?

So why choose these APIs? It’s simple, they just work, their easy to work with, flexible to fit into any application or platform and their tested.

Easy – The APIs are easy to implement because their simple REST calls.

Flexible – These APIs all work on whatever language, framework, or platform your choose. This means you can easily incorporate into your Windows, iOS, Android and Web apps using the tools and frameworks you already use and love (.NET, Python, Node.js, Xamarin, etc.).

Tested – Tap into the ever growing collection of APIs developed by the experts. You as developers can trust the quality and expertise built into each API by experts in their field from Microsoft Research, Bing, and Azure Machine Learning.

What’s also nice to know is that Microsoft Cognitive Services is now using the same terms as other Azure services. Under these new terms you as a Microsoft Cognitive Services customer, you own and can manage and delete your data.

Cognitive Services Real-World Applications

The following is a set of possible real-world application scenarios:

The Computer Vision API is able to extract rich information from images to categorize and process visual data and protect your users from unwanted content. Here, the API is able to tell us what the photo contains, indicate the most common colors, and lets us know that the content would not be considered inappropriate for users.

The Bing Speech API is capable of converting audio to text, understanding intent, and converting text back to speech for natural responsiveness. This case shows us that the user has asked for directions verbally, the intent has been extracted, and a map with directions provided.

Language Understanding Intelligent Service, known as LUIS, can be trained to understand user language contextually, so your app communicates with people in the way they speak. The example we see here demonstrates Language Understanding’s ability to understand what a person wants, and to find the pieces of information that are relevant to the user’s intent.

Knowledge Exploration Service adds interactive search over structured data to reduce user effort and increase efficiency. The Knowledge Exploration API example here demonstrates the usefulness of this API for answering questions posed in natural language in an interactive experience.

Bing Image Search API enables you to add a variety of image search options to your app or website, from trending images to detailed insights. Users can do a simple search, and this API scours the web for thumbnails, full image URLs, publishing website info, image metadata, and more before returning results.

These APIs are available as stand-alone solutions, or as past of the Cortana Intelligence Suite. These APIs can also be used in conjunction with the Microsoft Bot Framework.

Use Case: How Uber is Using Driver Selfies to Enhance Security

There is a use case where Uber is using Microsoft Cognitive Services to offer real-time ID check. Using the Face API, drivers are prompted to verify their identity by taking a selfie and then verifying that image with the one they have one file. The Face API is smart enough to recognize if you’re wearing glasses or a hat letting you take action and ask your users to remove and retry the verification process. Uber has made rides safer by giving their clients peace of mind that the drivers have been verified.

Dig Deeper into AI

If you’re interested in learning more about Microsoft AI then be sure to checkout these two websites:

http://azure.com/ai

http://aischool.microsoft.com

In my next post I’ll dig deeper into one of these APIs and walk through the code on how easily it is to incorporate into your applications.

Enjoy!

References

Microsoft Cognitive Services homepage

Microsoft Cognitive Services blog

Try Microsoft Cognitive Services

Cognitive Services Labs

Microsoft updates Cognitive Services terms

How Uber is using driver selfies to enhance security, powered by Microsoft Cognitive Services

Development

Microsoft Adaptive Cards 1.0 is now Available

Callon Campbell [MVP] on March 8, 2018

During the Windows Developer Day 2018 keynote, Kevin Gallo talked about Adaptive Cards and how they can be used to provide a flexible way to present your content and your data.

Adaptive Cards gives you the tools to create scale across any engagement surface.

Adaptive Cards was first announced at the Microsoft Build 2017 conference and has now come out of preview and is generally available as a 1.0 product. When Windows 10 Spring Creators Update is released, you will be able to use Adaptive Cards on the Windows Timeline along with other experiences like a bot, Skype, Notifications, Teams and so much more.

Enjoy!

References

Windows Developer Day 2018 Keynote

http://adaptivecards.io/

https://github.com/Microsoft/AdaptiveCards/releases/tag/v1.0

Development

Visual Studio 2017 (15.6) has new Update Experience

Callon Campbell [MVP] on March 8, 2018

After updating to Visual Studio 2017 (15.6) earlier today I noticed a minor update is out (15.6.1) and you will notice a new update experience as shown here. The updated dialog shows the current version, the update version and a link for the release notes:

This update (15.6.1) only takes a couple minutes to apply.

Enjoy!

Azure Events

Azure Developer Tour is Coming!

Callon Campbell [MVP] on March 5, 2018

The latest in compute, serverless and more – hosted by the Azure Advocates.

This is a a FREE event where you’ll learn about compute, serverless, storage, big data, artificial intelligence, machine learning and so much more…plus lunch is included (every developer loves a free lunch).

Cities

Toronto, On – April 3, 2018

Vancouver, BC – April 5, 2018

Washington, D.C. – April 6, 2018

Los Angeles, CA – April 9, 2018

Austin, TX – April 10, 2018

San Francisco, CA – April 12, 2018

Seattle, WA – April 12, 2018

In addition there will be University Tours for students and faculty.

Enjoy!

References

https://www.microsoftevents.com/profile/web/index.cfm?PKwebID=0x752173abcd&wt.mc_id=AID688794_owned_CESocial_Copy

Wrap up

References

Share with friends!

Like this:

What is the Face API?

Step 1 – Requirements

Step 2 – Subscribe to the Face API

Step 3 – Create new Application and reference the Face API

Step 4 – Configure the Face API Client Library

Step 5 – Upload images, detect faces, and show facial attributes

Wrap up

Resources

Share with friends!

Like this:

Share with friends!

Like this:

Creating a Lock using the Portal

Unlocking a Resource

Resources

Share with friends!

Like this:

Configuring your app for Application Insights

Accessing the Application Insights Telemetry from Visual Studio

References

Share with friends!

Like this:

References

Share with friends!

Like this:

Making AI Possible

Introducing Microsoft Cognitive Services

Why Use Microsoft Cognitive Services?

Cognitive Services Real-World Applications

Use Case: How Uber is Using Driver Selfies to Enhance Security

Dig Deeper into AI

References

Share with friends!

Like this:

References

Share with friends!

Like this:

Share with friends!

Like this:

Cities

References

Share with friends!

Like this: