Blog Microsoft Flow with Artificial Intelligence RSS Feed

Microsoft Flow with Artificial Intelligence

Dec 26, 2018

AI (Artificial Intelligence) based systems are trying to eliminate this up to some level of the human mistake by providing various services. Swift and efficient outputs are deliverable through AI enhanced systems in comparison to human outputs. Using these smart system, one is capable of providing decision with precision.

Microsoft has introduced AI services called  “Cognitive services”. There are various services categorized by Vision APIs, Speech APIs, Language APIs, and Search APIs which belong to Cognitive Services. These services can be integrated with many platforms very easily.

Thus, we can integrate above services with Microsoft Flow as it is well-known successor to the SharePoint development and prepare AI-Integrated Microsoft Flows which can perform smart job with perfection. Well in short and sweet sentence we can say, 

Microsoft FLOW + Cognitive Services = Surprisingly powerful tools!!!

Let’s dive into it in more details.

Generate Cognitive Service API Key and URL

To generate a Cognitive Service API Key and URL follow below steps:

  • Login into Azure account. This will show the list of services if one has already added any.

Azzure Account

  • Find the required service and click on Add. For Example: “Text Analysis”.

Text Analysis

  • The selected service will be added in the list of APIs.

Service Added

  • Note down any one value from the Endpoints and a key value either Key 1 or Key 2. These two noted values will be used in the later stage.

Cognitive Service Examples

Let’s go through 3 different Cognitive services via  Microsoft Flow. These services are integrated with Microsoft Flow.

  1. Text Analytics: It will detect sentiment of a sentence like positive, negative or neutral.
  2. Optical Character Recognition to Text: It will extract text from an uploaded image.
  3. Recognize Domain Specific Content: It will display name of celebrity from an uploaded pictureif there are any. 

Text Analytics

Flow requirement: Whenever a new feedback is added, an owner should get an email that whether the added feedback is positive, negative or neutral. Let’s assume, there is a list of user feedbacks as shown below.

Feedbacks

  • From the app launcher icon of the Office 365 portal, click Flow. This will redirect to My Flows page.

App launcher

  • Click + New > + Create from blank.

Create New

  • Click on Create from blank.

Create from blank

  • From the Triggers panel, find and select When an item is created.

When an item is created

  • Set values, Site Address = Site collection URL and List Name = Name of the list.

Set values

  • From the Action panel, find and select Detect Sentiment of Text Analytics API. Enter values:

a. Connection Name = Any name here

b. Account Key = Enter key value noted while adding this Service API i.e. either Key 1 or Key 2.

c. Site URL = Enter endpoint value noted while adding this Service API.

Then click “Create”.

Detect Sentiment

  • Once Service is connected, the input screen will appear. Inside the Text field select the Feedback variable from the Dynamic content panel.

Feedback variable

  • From the Action panel, find and select “Condition”.

Condition

Enter Values,

  • Choose a value = “Score” variable from Dynamic Content panel

  Dynamic Content

Here, Score is the output from “Text Analytics” action. 

The sentiment of feedback is depending on the score:

 o If Score < 0.3 then, feedback has a negative sentiment.

 o If Score =>0.3 and <=0.7 then, feedback has a neutral sentiment.

 o If Score >0.7 then, feedback has a positive sentiment.

  • For the negative sentiment, the condition must be as per below image.

Negative sentiment

  • If this condition is true, then send an email to an owner saying the submitted feedback has negative sentiment. Inside the If Yes section, find and add “Send an email” from the Action panel. Set below values:

To = Owner email 

Subject = Any text value

Body = Any text value conveying the negative sentiment.

Send an email

  • Now, if one wants to perform any action when this condition is false then an action can be placed inside If No section. If no action is required to perform then keep it blank.

Add an action

  • Similarly, for positive sentiment, add the condition and the send email action as per below image.

add the condition

  • For neutral sentiment, add the condition in advanced mode and the send email action. 
  • In this case, the condition is a combination of two different conditions and syntax will be like this:

Condition = ‘@and(greaterOrEquals(body('Detect_Sentiment')?[‘score’],

0.3),lessOrEquals(body('Detect_Sentiment')?[‘score’], 0.7))’

Select Condition

  • Flow is ready to provide the sentiment of a submitted feedback. See below screenshots for these 3 different sentiments. 

Negative Sentiment Feedback:

Negative Sentiment Feedback

Positive Sentiment Feedback:

Positive Sentiment Feedback

Neutral Sentiment Feedback:

Neutral Sentiment Feedback

Optical Character Recognition to Text:

Flow requirement: Whenever a new image is uploaded, a flow will extract text from the image and an email will be sent to an owner with that text.  Let’s say, there is an Images Document Library to upload images as shown below.

Images Document Library

  1. Follow 1, 2 and 3 steps from Text Analytics example. Then start below steps,
  2. From the Trigger panel, find and add trigger When a file is created (properties only).

    Set values: 

    • Site Address = Site collection URL 
    • Library Name = Name of library where images will be uploaded. 
    • Folder = Empty (i.e. Root Folder)

    Set values

  3. From the Action panel, find add action Get content.

     

    Set values:

    • Site Address = Site collection URL
    • File Identifier = Select “Identifier” variable from Dynamic content panel

    Get file content

  4. From the Action panel, find and add service “Optical Character OCR to Text”. If this service asks for Account Key and Site URL values, then provide the respective noted values of Computer Vision API. 

    • Connection Name = Any name here
    • Account Key = Enter key value noted while adding this Service API i.e. either Key 1 or Key 2.
    • Site URL = Enter endpoint value noted while adding this Service API.
  5. After providing the above values, set below values:

    • Image Source = Image Content
    • Image Content = File Content variable from Dynamic Content panel

    Optical Character OCR

  6. The above action will extract text from the uploaded image. 
  7. From Action panel, find and add Send an email action, and set values:

    To = Owner email

    Subject = Any text value

    Body = Any text value with variable Detected Text from the Dynamic Content panel. “Detected Text” is an output of OCR to Text service.

    Send an email action

  8. Save the flow and upload an image in the Images Document Library. For example: below image is uploaded in the library.

    Upload

  9. The below email is sent to an owner.

    New Image Uploaded

 

Recognize Domain Specific Content: 

Well, this is an interesting service. It can rectify names of celebrities in an image if there are any!! Let see how it works.

We will use the same ImagesDocumentLibrary.

Flow requirement: Whenever a new image is uploaded, flow should rectify celebrity names from the image and an email will be sent to the owner with those names.

  1. Follow 1, 2 and 3 steps from Optical Character Recognition to Text example. 
  2. From Action panel, find and add “Recognize Domain Specific Content”. If this service asks for Account Key and Site URL values, then provide the respective noted values of Computer Vision API.

       a. Connection Name = Any name here

      b. Account Key = Enter key value noted while adding this Service API i.e. either Key 1 or Key 2.

      c. Site URL = Enter endpoint value noted while adding this Service API.  

  3. After providing above values, set below values:

    • Domain Model = celebrities
    • Image Source = Image Content
    • Image Content = File Content variable form Dynamic Content panel

    Recognize Domain Specific Content

  4. The output of the above action will be an array of celebrity names. 
  5. From Action panel, find and add “Send an email” action. Set values,

    • To = Owner email
    • Subject = Any text value
    • Body = Any text value with Celebrity Name variable from Dynamic Content Panel. Here, apply to each action will be automatically added which iterate through each celebrity.

    Apply to each

  6. Save the flow and upload an image in the Images Document Library. For example: below image is uploaded in the library.

    Steve Jobs

  7. The below email is sent to an owner.  

    New celebrity image uploaded

 

Conclusion

Microsoft has provisioned many Cognitive Services apart from the above 3 services. One can explore them based on his/her requirements. Many complex business requirements are achievable in very less time with efficiency. The advantage of these services is that they can be integrated with many popular platforms like Twitter, MailChimp, Dynamics 365 etc.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

About Tatvasoft

TatvaSoft is a CMMi Level 3 and Microsoft Gold Certified Software Development Company offering custom software development services on diverse technology platforms, like Microsoft, SharePoint, Biztalk, Java, PHP, Open Source, BI, Big Data and Mobile.

Follow Us