Information in this document, including URL and other Internet Web site references, is subject to change without notice. Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, e-mail address, logo, person, place or event is intended or should be inferred. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.
Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.
The names of manufacturers, products, or URLs are provided for informational purposes only, and Microsoft makes no representations and warranties, either expressed, implied, or statutory, regarding these manufacturers or the use of the products with any Microsoft technologies. The inclusion of a manufacturer or product does not imply endorsement of Microsoft of the manufacturer or product. Links may be provided to third-party sites. Such sites are not under the control of Microsoft and Microsoft is not responsible for the contents of any linked site or any link contained in a linked site, or any changes or updates to such sites. Microsoft is not responsible for webcasting or any other form of transmission received from any linked site. Microsoft is providing these links to you only as a convenience, and the inclusion of any link does not imply endorsement of Microsoft of the site or the products contained therein.
Cognitive Services and deep learning hands-on lab step-by-step
Abstract and learning objectives
In this workshop, you will learn to combine both pre-built artificial intelligence (AI) (in the form of various Cognitive Services) with custom AI (in the form of services built and deployed with Azure Machine Learning services). You will learn to create intelligent solutions atop unstructured text data by designing and implementing a text analytics pipeline. You will also learn how to build a binary classifier using a simple neural network that can be used to classify the textual data. Also, you will learn how to deploy multiple kinds of predictive services using Azure Machine Learning and learn to integrate with the Computer Vision API and the Text Analytics API from Cognitive Services.
Along the way, you will get to consider the following technologies and services:
Azure Machine Learning services
Cognitive Services
Computer Vision API
Text Analytics API
TensorFlow
Overview
In this workshop, you will help Contoso Ltd. Build a proof of concept that shows how they could build a solution that amplifies the claims processing capabilities of their agents.
Solution architecture
The high-level architecture of the solution is illustrated in the diagram. The lab is performed within the context of a Jupyter Notebook running within a Data Science VM on Azure. Various notebooks are built to test the integration with the Cognitive Services listed, to train customer ML services and to integrate the results in a simple user interface that shows the result of processing the claim with all of the AI services involved.
The High-level architectural solution begins with a Claim, which points to Jupyter notebook. Jupyter then points to Computer Vision, Text Analytics, and Containerized Services, which includes a Classification Service and a Summary Service that both process claim text.
Requirements
Microsoft Azure subscription must be pay-as-you-go or MSDN
Trial subscriptions will not work
Exercise 1: Setup Azure Machine Learning accounts
Duration: 45 minutes
In this exercise, you will setup your Azure Machine Learning Experimentation and Model Management Accounts and get your project environment setup.
Task 1: Provision Azure Machine Learning Experimentation service
Navigate to the Azure Portal.
Select Create a resource.
Select AI + Cognitive Services and then select Machine Learning Experimentation
On the ML Experimentation blade, provide the following:
Experimentation account name: provide a name for your experimentation account.
Subscription: select your Azure subscription.
Resource group: select the mcw-ai-lab resource group you previously created.
Location: select the region nearest to where you deployed your Data Science VM. It’s OK if they are not in exactly the same region, but try to select a region that is close to minimize latency.
Number of seats: leave at 2.
Storage account: select create new and provide a unique name for the new storage account.
Workspace for Experimentation account: provide a unique name for the workspace.
Assign owner for the workspace: leave the owner assigned to you.
Create Model Management account: leave checked.
Account name: provide a name for your model management account.
Model Management pricing tier: select the S1 pricing tier.
Select Create to provision the Experimentation and Model Management Service. The deployment should take about 2 minutes.
When the deployment completes, navigate to your mcw-ai-lab resource group and confirm that you see an instance of Machine Learning Experimentation and Machine Learning Model Management.
Task 2: Create the Azure Machine Learning project
Connect to the labvm via RDP. If you stopped the VM, remember to Start it up again before attempting to connect.
From the Start menu, launch Azure Machine Learning Workbench.
Select Sign in with Microsoft.
Sign in with the credentials you used when creating the Experimentation Service in the Azure Portal.
After successfully signing in, the Workbench interface should appear, listing the experimentation workspace that you created.
From the File Menu, select New Project…
In the New Project blade that appears, provide the following:
Project name: mcw-ai-lab
Project directory: C:\HOL
Project description: leave blank
Vistualstudio.com GIT Repository URL: leave blank
Selected workspace: select the ML Experimentation Workspace you created.
In the Search Project Templates, enter TDSP and select the item called TDSP Template.
Select Create.
The template will download, and a few moments you should see the TDSP project dashboard.
Task 3: Install dependencies
The tasks that follow depend on Python libraries like nltk and gensim. The following steps ensure you have these installed in your environment.
From the File menu of Workbench, select Open Command Prompt.
Run the following command to install nltk:
pip install nltk
NLTK should install, with a message similar to the following:
NLTK is a rich toolkit with modular components, many of which are not installed by default. To install all the components, run the python shell by entering python at the command prompt:
python
Within the python shell, run the following two lines:
import nltk
nltk.download('all')
The downloader will take about 5 minutes to complete. Once it is finished, exit the python shell by entering:
exit()
Run the following command to install genism:
pip install gensim
Next, download a pre-built Jupyter Notebook that you will step through to understand the process used to summarize the text of claims documents. In the Firefox browser on your VM, navigate to the following (note that the URL is case sensitive). Note, if using IE you will need to modify the default security settings, which prevent files from being downloaded.
In the command prompt, enter the following and press enter to launch the Jupyter Notebook:
jupyter notebook
In a few moments, you should be prompted for which browser to use to open the link, select Firefox.
The Jupyter Notebook interface should appear in the browser, listing the contents of your project folder.
Select the code folder.
Select 01_data_acquisition_and_understanding.
Select the Upload button.
Open the Summarize.ipynb notebook and follow the instructions within it.
Exercise 2: Deploy the Summarizer as a Service
Duration: 45 minutes
In this exercise you will create and deploy a web service that uses a pre-trained model to summarize long paragraphs of text.
Task 1: Deploy your ACS cluster
Within Workbench, from the File menu, select Open Command Prompt.
Create the cluster environment by running the following command, replacing the values indicated in angle brackets with appropriate values. This will create new resources groups for the cluster.
For <environment name> enter mcwailabenv, or something similar. This value can only contain lowercase alphanumeric characters.
For location, use eastus2, westcentralus, australiaeast, westeurope, or southeastasia, as those are the only acceptable values at this time.
az ml env setup -c -n <environment name> --location <e.g. eastus2>
When prompted, copy the URL presented and sign in using your web browser.
Enter the code provided in the command prompt.
Select Continue.
Sign in with your Azure Credentials.
Return to the command prompt, which should automatically update after you log in.
At the “Subscription set to <subscription name>” prompt, enter Y if the subscription name is correct, or N to select the subscription to use from a list.
It will take 10-20 minutes for your ACS cluster to be ready. You can periodically check on the status by running the command shown in the output to the previous step, which is of the form:
az ml env show -g <resourceGroupName> -n <clusterName>
Once the environment has successfully provisioned (the Provisioning State in the above command will read “Succeeded”), run the other command provided in step 8, which is of the form:
az ml env set -g <resourceGroupName> -n <clusterName>
This will set the context of the command line to target this environment.
Finally, set the model management account to be used by the command line, to be the one you created previously (mcw-ai-lab-model-mgmt). Run the following command, replacing the values indicated in angle brackets with appropriate values.
For <acctname>, enter the name of the Machine Learning Model Management resource in your mcw-ai-lab resource group.
For <resourcegroupname>, use your mcw-ai-lab resource group name.
az ml account modelmanagement set -n <acctname> -g <resourcegroupname>
Task 2: Set Visual Studio Code as the project IDE in Workbench
Within Workbench, from the File menu, select Configure Project IDE.
In the Configure IDE blade that appears, set the following properties:
Name: Visual Studio Code
Executable Path: C:\Program Files\Microsoft VS Code\Code.exe
Select OK.
Launch Visual Studio Code for the project by selecting Open Project (Visual Studio Code) from the File menu.
You are now ready to author service script.
Task 3: Create the Summarization service
Visual Studio Code will open against the project directory.
In the tree view, expand code and then right-click 03_deployment and select New File.
For the file name, enter summarizer_service.py and press Enter.
Paste the contents of this script into your summarizer_service.py. Take a moment to review the script, as it is effectively the same code you were running in the Jupyter notebook, except that is has been modified to follow the format required by services in Azure Machine Learning. The init method is called once per container by the Azure Machine Learning infrastructure when the service is deployed. It is here that we need to load all of the modules required by NLTK in the call to nltk.download. The run method is where any scoring (or in our case summarization) activity takes place.
Next, we need to capture the dependencies for the modules used by the script. These are declared in a conda environment file, which you can generate from an environment or create by hand. In this case, we will edit the default conda environment provided by the TDSP project by hand, and add a configuration that will pip install gensim as required by our script. To do this, in Visual Studio Code, expand, aml_config and open conda_dependencies.yml.
At the last line, under azure-ml-api-sdk add another line with -gensim to the pip configuration. You should also add entries for tensorflow and tflearn, which we will need later in the lab. Your final configuration should look as follows:
name: project_environment
dependencies:
- python=3.5.2
- scikit-learn
- pip:
# The API for Azure Machine Learning Model Management Service.
# Details: https://github.com/Azure/Machine-Learning-Operationalization
- azure-ml-api-sdk==0.1.0a11
- gensim
- tensorflow
- tflearn
Save the file. When we go to create the image in a later step, this file will be included with command.
Next, create an empty file called dummy_model.bin in the 03_deployment folder. In this case, we don’t have a model to deploy with this service, but we still need to provide one to the CLI as we will see in a moment. An empty file will do.
Task 4: Deploy the Summarization service
Return to the Workbench and use the File menu to open another command prompt.
At the command prompt, change directories to the code\03_deployment directory by executing the following command:
cd code\03_deployment
You can deploy the service using a single command (which orchestrates the multiple steps of creating a docker manifest, creating a docker image, and deploying a container instance from the image). The command needs to refer to all the components required for the service including the dummy model file, the service script, the conda dependencies and the runtime to use (python in this case). Run the following command to deploy the summarizer service:
az ml service create realtime -n summarizer -c ..\..\aml_config\conda_dependencies.yml -m dummy_model.bin -f summarizer_service.py -r python
In the Command Prompt window, the previous command and its output display. At this time, we are unable to capture all of the information in the command prompt window. Future versions of this course should address this.
Notice in the output of the preceding command, you are provided with instructions (third line from last) on how you can invoke the deployed service using the CLI. Try executing the following command (modify the Service ID of you service as indicated in the previous command output):
az ml service run realtime -i summarizer.[mcwailab-xyz.location] -d "I was driving down El Camino and stopped at a red light. It was about 3pm in the afternoon. The sun was bright and shining just behind the stoplight. This made it hard to see the lights. There was a car on my left in the left turn lane. A few moments later another car, a black sedan pulled up behind me. When the left turn light changed green, the black sedan hit me thinking that the light had changed for us, but I had not moved because the light was still red. After hitting my car, the black sedan backed up and then sped past me. I did manage to catch its license plate. The license plate of the black sedan was ABC123."
In the Command Prompt window, the previous text displays.
If you get a summary back, your service is working! Try calling the service with other text and observe the summary returned. Note that the service tries to build a summary of about 30 words, so if you provide too short a text, an empty summary will be returned.
Finally, in a notepad or other location take note of the full Service ID (e.g., summarizer.mcwailab-xyz.location) and the authorization key which you will need later in the lab. To get the authorization key for your deployed service, run the following command and take note of the PrimaryKey value in the output:
az ml service keys realtime -i summarizer.[mcwailab-xyz.location]
In the Command Prompt window, the previous command and its output display. At this time, we are unable to capture all of the information in the command prompt window. Future versions of this course should address this.
Exercise 3: Applying TensorFlow
Duration: 60 minutes
In this exercise, you use TensorFlow to construct and train a simple deep neural network classification model that will classify claim text as belonging to a home insurance claim or an automobile claim. You will then deploy this trained model as a web service.
Task 1: Prepare TensorFlow
Return to your RDP session to the Data Science VM.
Switch to the command prompt that is running the Jupyter Notebook command and press Control + Break. This will stop the Jupyter Notebook process while you update TensorFlow.
From the command line run:
pip install tensorflow
In a few moments, the install should complete, and you should see output ending similar to the following:
We will be using the TFLearn library which sits atop TensorFlow. To install it run:
pip install tflearn
In the Command Prompt window, the installation progress and the success message displays. At this time, we are unable to capture all of the information in the command prompt window. Future versions of this course should address this.
Run “Jupyter Notebook” to re-start the process.
You should now be ready to use TensorFlow on your Data Science VM.
Task 2: Train and deploy the TensorFlow model
Return to your RDP session to the Data Science VM.
Download the TensorFlow notebook, text analytics helper module and sample data from the following link:
Extract this zip and copy the contents to C:\HOL\mcw-ai-lab\code\02_modeling.
Return to the instance of the Jupyter Notebook home that should be open in your browser.
Select the code folder, 02_modeling. You should see a folder listing similar to the following. Select Claim Classification.ipynb.
The Claim Classification notebook will appear. Step through this notebook to read how the data is prepared and the neural network model is trained. Be sure to execute each cell as you get to it.
When you have finished executing the notebook, some model files will have been produced. Using File Explorer, navigate to C:\HOL\mcw-ai-lab\code\02_modeling, you should see the three new files (each beginning with claim_classifier.tfl).
Copy these three files and paste them under C:\HOL\mcw-ai-lab\code\03_deployment\claim_class_service. You are copying these over so they can be used by the predictive web service we will deploy.
Next, download the supporting files for the claim_class_service from:
Extract the files and copy them into C:\HOL\mcw-ai-lab\code\03_deployment\claim_class_service.
Return to the instance of the Jupyter Notebook home that should be open in your browser.
Select the code folder, 03_deployment and then claim_class_service.
Open claim_class_service.py. Observe that the code it uses is like what you ran in the Claim Classification notebook, only formatted to fit the structure of an Azure Machine Learning web service (with init and run methods).
Next, you will deploy this service. Switch to your command line window and navigate to C:\HOL\mcw-ai-lab\code\03_deployment\claim_class_service.
Run the following command in the context of the claim_class_service folder to deploy the service:
az ml service create realtime -n claimclassifier -c ..\..\..\aml_config\conda_dependencies.yml -m claim_classifier.tfl.meta -f claim_class_service.py -r python -d claim_classifier.tfl.data-00000-of-00001 -d claim_classifier.tfl.index -d claims_text.txt -d textanalytics.py -d contractions.py
In the Command Prompt window, the previous command and its output displays. At this time, we are unable to capture all of the information in the command prompt window. Future versions of this course should address this.
Next, test the deployed service by running the following command (substitute the values of the Service ID as indicated in the last line of the previous):
az ml service run realtime -i claimclassifier.[mcwailab-xyz.location] -d "A tornado ripped through my home."
In the Command Prompt window, the previous command and its output display. At this time, we are unable to capture all of the information in the command prompt window. Future versions of this course should address this.
Recall the classifier will return 1 if the text is classified as related to a car insurance claim and 0 if the claim pertains to a home insurance claim. Try submitting a few different sentences to the service.
Next, in a notepad or other location take note of the full Service ID (e.g., claimclassifier.mcwailab-xyz.location) and the authorization key which you will need later in the lab. To get the authorization key for your deployed service, run the following command and take note of the PrimaryKey value in the output:
az ml service keys realtime -i claimclassifier.[mcwailab-xyz.location]
In the Command Prompt window, the previous command and its output display. At this time, we are unable to capture all of the information in the command prompt window. Future versions of this course should address this.
Finally, run the following command to retrieve the IP address of your claimclassifier and summarizer services, and note the value in notepad or other location for use later in the lab. The IP address will be the same for both services.
az ml service usage realtime -i claimclassifier.[mcwailab-xyz.location]
In the Command Prompt window, the IP address displays.
Exercise 4: Completing the solution
Duration: 45 minutes
In this exercise, you perform the final integration with the Computer Vision API and the Text Analytics API along with the Azure Machine Learning Services you previously deployed to deliver the completed proof of concept solution.
Task 1: Deploy the Computer Vision API
Navigate to the Azure Portal in your browser.
Select Create a resource.
Select AI + Cognitive Services and then Computer Vision API.
On the Create blade, provide the following:
Name: provide a unique name for this instance.
Subscription: select your Azure subscription.
Location: select a location nearest your other deployed services.
Pricing tier: select S1.
Resource group: select the existing mcw-ai-lab resource group.
Select Create.
When the notification appears that the deployment succeeded, select Go to resource.
Select Keys and then copy the value of Key 1 into notepad or something similar as you will need this value later in the lab.
Select Overview and copy the value of Endpoint from the Essentials panel. Store this value in notepad or something similar as you will need this value later in the lab.
Task 2: Deploy the Text Analytics API
Navigate to the Azure Portal in your browser.
Select Create a resource.
Select AI + Cognitive Services and then Text Analytics API.
On the Create blade, provide the following:
Name: provide a unique name for this instance.
Subscription: select your Azure subscription.
Location: select a location nearest your other deployed services.
Pricing tier: select S0.
Resource group: select the existing mcw-ai-lab resource group.
Select Create.
When the notification appears that the deployment succeeded, select Go to resource.
Select Keys and then copy the value of Key 1 in to notepad or something similar as you will need this value later in the lab.
Select Overview and copy the value of Endpoint from the Essentials panel. Store this value in notepad or something similar as you will need this value later in the lab.
Task 3: Completing the solution
Return to your RDP session to the Data Science VM.
Extract the contents of this zip file to C:\HOL\mcw-ai-lab\code\03_deployment.
Return to the instance of the Jupyter Notebook home that should be open in your browser.
Select the code folder, 03_deployment. You should see a folder listing like the following. Select Cognitive Services.ipynb.
Follow the steps within the notebook to complete the lab and view the result of combining Cognitive Services with your Azure Machine Learning Services.
After the hands-on lab
Duration: 5 minutes
To avoid unexpected charges, it is recommended you clean up all of your lab resources when you complete the lab.
Task 1: Clean up lab resources
Navigate to the Azure Portal and locate the Resource Groups you created for this lab
mcw-ai-lab
mcwailabenv (note there are two resources groups starting with this name, so delete both)
Select Delete resource group from the command bar.
In the confirmation dialog that appears, enter the name of the resource group and select Delete.
Wait for the confirmation that the Resource Group has been successfully deleted. If you don’t wait, and the delete fails for some reason, you may be left with resources running that were not expected. You can monitor using the Notifications dialog, accessible from the Alarm icon.
When the Notification indicates success, the cleanup is complete.
You should follow all steps provided after attending the Hands-on lab.
