|

|  How to Integrate Microsoft Azure Cognitive Services with WhatsApp

How to Integrate Microsoft Azure Cognitive Services with WhatsApp

January 24, 2025

Learn to seamlessly connect Microsoft Azure Cognitive Services with WhatsApp, enhancing your communication abilities with AI-driven insights and features.

How to Connect Microsoft Azure Cognitive Services to WhatsApp: a Simple Guide

 

Set Up Microsoft Azure Cognitive Services

 

  • Go to the Microsoft Azure portal and sign in with your Azure credentials.
  •  

  • Search for "Cognitive Services" in the search bar and select it from the list.
  •  

  • Click on "Create" to start a new instance. Fill in the required details like Subscription, Resource group, and the name of your service.
  •  

  • Choose the type of Cognitive Service you want to use (e.g., Text Analytics, Language Understanding). Note the API key and endpoint URL as these will be used later.

 

Set Up WhatsApp API

 

  • Sign up for a service that provides a WhatsApp Business API, such as Twilio or MessageBird, as you can’t directly interact with WhatsApp from Azure.
  •  

  • Configure your WhatsApp Business account through the service provider’s console, ensuring that you have the API key and URL for sending messages.

 

Create a Bridge Service

 

  • Establish a local or cloud-based application that acts as a bridge between Azure Cognitive Services and the WhatsApp API. You can use Node.js or Python as the programming language for handling HTTP requests and responses.
  •  

  • Use the following Node.js code as an example to set up your project:

 

const express = require('express');
const axios = require('axios');
const app = express();

app.use(express.json());

const azureCognitiveServiceEndpoint = 'YOUR_AZURE_SERVICE_ENDPOINT';
const azureApiKey = 'YOUR_AZURE_API_KEY';
const whatsappEndpoint = 'YOUR_WHATSAPP_ENDPOINT';
const whatsappApiKey = 'YOUR_WHATSAPP_API_KEY';

app.post('/webhook', async (req, res) => {
    try {
        const message = req.body.message; // WhatsApp message
        const cognitiveResponse = await axios.post(azureCognitiveServiceEndpoint, {body: message}, {
            headers: {'Ocp-Apim-Subscription-Key': azureApiKey}
        });

        const analyzedMessage = cognitiveResponse.data;
        await axios.post(whatsappEndpoint, {
            headers: {'Authorization': `Bearer ${whatsappApiKey}`},
            message: analyzedMessage
        });

        res.status(200).send('Message sent to WhatsApp');
    } catch (error) {
        console.error('Error processing request:', error);
        res.status(500).send('Error');
    }
});

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
    console.log(`Server running on port ${PORT}`);
});

 

Deploy and Test the Service

 

  • Deploy your application to a cloud service like Heroku, AWS, or Azure App Services to make it publicly accessible.
  •  

  • Set up a webhook through your WhatsApp API provider’s dashboard. Ensure this webhook URL points to your deployed application.
  •  

  • Send a test message from your WhatsApp to verify that the Cognitive Service processes it correctly and sends a response back via WhatsApp.

 

Omi Necklace

The #1 Open Source AI necklace: Experiment with how you capture and manage conversations.

Build and test with your own Omi Dev Kit 2.

How to Use Microsoft Azure Cognitive Services with WhatsApp: Usecases

 

Advanced Customer Service using Azure Cognitive Services and WhatsApp

 

  • Seamless Integration: Leverage Azure Cognitive Services to understand and process customer inquiries received via WhatsApp. Integrating these two platforms provides a cohesive experience, allowing AI to handle a variety of customer needs efficiently.
  •  

  • Sentiment Analysis: Utilize Azure's Text Analytics API to gauge customer sentiment in real-time. This allows businesses to adapt their customer interaction strategies, ensuring a tailored and positive customer service experience.
  •  

  • Language Translation: Use Azure Translator to bridge communication gaps with customers from different linguistic backgrounds. This includes automatic detection and translation of messages, improving accessibility and service reach.
  •  

  • Automated Customer Support: Employ Azure’s Speech and Language APIs to build intelligent bots that manage routine inquiries on WhatsApp, freeing human agents to focus on more complex or high-touch situations.
  •  

  • User Authentication: Implement Azure Face Recognition to add a layer of security by authenticating users via images sent through WhatsApp, thereby enhancing trust and reducing fraud.
  •  

  • Data-Driven Insights: Analyze interaction data captured over WhatsApp through Azure's Power BI, enabling businesses to derive actionable insights and continuously improve their customer support strategies.

 

# Example of an API call to analyze sentiment of a WhatsApp message
curl -X POST "https://<your-custom-subdomain>.cognitiveservices.azure.com/text/analytics/v3.0/sentiment" 
     -H "Ocp-Apim-Subscription-Key: <your-subscription-key>" 
     -H "Content-Type: application/json" 
     --data-ascii "{\"documents\":[{\"id\":\"1\",\"language\":\"en\",\"text\":\"I love the new updates on your app!\"}]}"

 

 

Intelligent Healthcare Assistance using Azure Cognitive Services and WhatsApp

 

  • Patient Query Handling: With Azure's Natural Language Processing tools, healthcare providers can efficiently manage and respond to patient inquiries received through WhatsApp. This ensures quick and personalized assistance, leading to improved patient satisfaction and care outcomes.
  •  

  • Real-Time Language Support: Azure Translator aids in breaking language barriers, allowing healthcare providers to communicate with patients in their native language via WhatsApp. This enhances accessibility and ensures all patients receive equitable healthcare services.
  •  

  • Symptom Checker and Triage: Implement Azure's Language Understanding (LUIS) to build a virtual triage tool on WhatsApp that interprets symptom descriptions, helping to prioritize medical attention based on urgency and severity.
  •  

  • Secure Patient Authentication: Azure Face API can authenticate patient identities through selfies shared on WhatsApp, thus safeguarding sensitive medical information and streamlining access to healthcare services.
  •  

  • Appointment Scheduling: Leverage Azure Bot Service coupled with WhatsApp to create an automated system for scheduling and confirming appointments, reducing administrative burdens and enhancing healthcare facility efficiency.
  •  

  • Health Monitoring and Alerts: Utilize Azure's Machine Learning capabilities to analyze patient data collected via IoT devices, delivering timely alerts through WhatsApp about abnormal health metrics, which can contribute to preventive healthcare delivery.

 

# Example API call to translate a medical consultation message in WhatsApp
curl -X POST "https://<your-custom-subdomain>.cognitiveservices.azure.com/translator/text/v3.0/translate?to=es" 
     -H "Ocp-Apim-Subscription-Key: <your-subscription-key>" 
     -H "Content-Type: application/json" 
     --data-ascii "{\"texts\":[{\"text\":\"I'm experiencing mild chest pain.\"}]}"

 

Omi App

Fully Open-Source AI wearable app: build and use reminders, meeting summaries, task suggestions and more. All in one simple app.

Github →

Order Friend Dev Kit

Open-source AI wearable
Build using the power of recall

Order Now

Troubleshooting Microsoft Azure Cognitive Services and WhatsApp Integration

How do I integrate Azure Language Understanding with WhatsApp?

 

Setup Azure Language Understanding

 

  • Sign into the Azure portal and create a new Language Understanding resource.
  •  

  • Note down the endpoint and subscription key, as you'll need them for integration.

 

WhatsApp Integration

 

  • Use Twilio or another service to access WhatsApp's API. Create an account and get the API credentials.
  •  

  • Set up a webhook URL where Twilio will send incoming messages.

 

Create a Node.js Server

 

  • Install dependencies: express for the server and axios for HTTP requests.

 

npm install express axios

 

Handle Messages

 

  • Set up an Express server to process WhatsApp messages.

 

const express = require('express');
const axios = require('axios');
const app = express();

app.post('/webhook', async (req, res) => {
    const message = req.body.Body;
    const response = await axios.post('https://<your-luis-endpoint>', { query: message },
                                       { headers: { 'Ocp-Apim-Subscription-Key': '<your-key>' }});
    // Process LUIS response
    res.send();
});

app.listen(3000);

 

Deploy & Test

 

  • Deploy your server and configure the webhook URL on Twilio. Test by sending messages to your WhatsApp number.

 

Why is my Azure Bot Framework not responding on WhatsApp?

 

Check WhatsApp Channel Configuration

 

  • Ensure your WhatsApp Channel in Azure is configured correctly with the necessary credentials and endpoints from your WhatsApp Business Account.

 

Verify Messaging Endpoint

 

  • Confirm the messaging endpoint for your bot is correct in the Azure portal. Check for typos in the endpoint URL.

 

Inspect Bot Code

 

  • Review the bot's message handling logic for any exceptions or infinite loops preventing response. Consider adding logging.

 

async def on_message_activity(turn_context: TurnContext):
    if turn_context.activity.type == ActivityTypes.message:
        await turn_context.send_activity("Hello World!")

 

Review Azure Logs

 

  • Check the Azure App Service logs and Application Insights for errors or warnings when the bot should have responded.

 

Test with Direct Line

 

  • Temporarily connect using Direct Line or Web Chat for troubleshooting to see if the issue is isolated to WhatsApp.

 

az webapp log tail --name <appname> --resource-group <group>

 

How to set up Azure Text Analytics for WhatsApp messages?

 

Set Up Azure Text Analytics

 

  • Sign up for Microsoft Azure and create a Text Analytics resource.
  •  

  • Get the API key and endpoint URL from the Azure portal.

 

Integrate with WhatsApp

 

  • Use Twilio API for WhatsApp message handling. Sign up for Twilio, set up a Messaging Service, and link WhatsApp.
  •  

  • Get your Twilio Account SID, Auth Token, and WhatsApp-enabled phone number.

 

Code Implementation

 

from azure.ai.textanalytics import TextAnalyticsClient
from azure.core.credentials import AzureKeyCredential
from twilio.rest import Client

# Initialize clients
azure_client = TextAnalyticsClient(endpoint="YourAzureEndpoint", 
                                   credential=AzureKeyCredential("YourAPIKey"))
twilio_client = Client("YourTwilioSID", "YourTwilioAuthToken")

# Receive and analyze WhatsApp messages
def analyze_message(message):
    documents = [message]
    response = azure_client.analyze_sentiment(documents)
    return response[0].sentiment

message = "Sample WhatsApp message"
sentiment = analyze_message(message)
print(f"Sentiment: {sentiment}")

 

Don’t let questions slow you down—experience true productivity with the AI Necklace. With Omi, you can have the power of AI wherever you go—summarize ideas, get reminders, and prep for your next project effortlessly.

Order Now

Join the #1 open-source AI wearable community

Build faster and better with 3900+ community members on Omi Discord

Participate in hackathons to expand the Omi platform and win prizes

Participate in hackathons to expand the Omi platform and win prizes

Get cash bounties, free Omi devices and priority access by taking part in community activities

Join our Discord → 

OMI NECKLACE + OMI APP
First & only open-source AI wearable platform

a person looks into the phone with an app for AI Necklace, looking at notes Friend AI Wearable recorded a person looks into the phone with an app for AI Necklace, looking at notes Friend AI Wearable recorded
a person looks into the phone with an app for AI Necklace, looking at notes Friend AI Wearable recorded a person looks into the phone with an app for AI Necklace, looking at notes Friend AI Wearable recorded
online meeting with AI Wearable, showcasing how it works and helps online meeting with AI Wearable, showcasing how it works and helps
online meeting with AI Wearable, showcasing how it works and helps online meeting with AI Wearable, showcasing how it works and helps
App for Friend AI Necklace, showing notes and topics AI Necklace recorded App for Friend AI Necklace, showing notes and topics AI Necklace recorded
App for Friend AI Necklace, showing notes and topics AI Necklace recorded App for Friend AI Necklace, showing notes and topics AI Necklace recorded

OMI NECKLACE: DEV KIT
Order your Omi Dev Kit 2 now and create your use cases

Omi Dev Kit 2

Endless customization

OMI DEV KIT 2

$69.99

Make your life more fun with your AI wearable clone. It gives you thoughts, personalized feedback and becomes your second brain to discuss your thoughts and feelings. Available on iOS and Android.

Your Omi will seamlessly sync with your existing omi persona, giving you a full clone of yourself – with limitless potential for use cases:

  • Real-time conversation transcription and processing;
  • Develop your own use cases for fun and productivity;
  • Hundreds of community apps to make use of your Omi Persona and conversations.

Learn more

Omi Dev Kit 2: build at a new level

Key Specs

OMI DEV KIT

OMI DEV KIT 2

Microphone

Yes

Yes

Battery

4 days (250mAH)

2 days (250mAH)

On-board memory (works without phone)

No

Yes

Speaker

No

Yes

Programmable button

No

Yes

Estimated Delivery 

-

1 week

What people say

“Helping with MEMORY,

COMMUNICATION

with business/life partner,

capturing IDEAS, and solving for

a hearing CHALLENGE."

Nathan Sudds

“I wish I had this device

last summer

to RECORD

A CONVERSATION."

Chris Y.

“Fixed my ADHD and

helped me stay

organized."

David Nigh

OMI NECKLACE: DEV KIT
Take your brain to the next level

LATEST NEWS
Follow and be first in the know

Latest news
FOLLOW AND BE FIRST IN THE KNOW

thought to action

team@basedhardware.com

company

careers

events

invest

privacy

products

omi

omi dev kit

personas

resources

apps

bounties

affiliate

docs

github

help