AWS AppSync – HTTP Resolver

AWS AppSync is an enterprise level, fully managed GraphQL service with real-time data synchronization and offline programming features.
AWS AppSync automatically updates the data in web and mobile applications in real time, and updates data for offline users as soon as they reconnect. AWS AppSync makes it easy to build collaborative mobile and web applications that deliver responsive, collaborative user experiences.

AWS AppSync enables you to use supported data sources (AWS Lambda, Amazon DynamoDB, or Amazon Elasticsearch Service) to perform various operations, in addition to any arbitrary HTTP endpoints to resolve GraphQL fields.

Here you can read more about AWS AppSync:

AppSync now supports HTTP endpoints as data sources: HTTP resolver.
It enables customers to use their existing backend services that use REST APIs with AppSync to leverage the power of GraphQL interfaces.

If you are not familiar with GraphQL, I suggest you these resources:

In this post we are going to see how to create a new AppSync API, a new HTTP data source, a new HTTP resolver and how to run a simple GraphQL statement to query our backend Rest API service.

To run the example I used Python 3.6.3. You need the Python AWS SDK Boto3.

If you have an older version of Boto3, please update it (you need at least version boto3-1.7.59).

Given our REST Api endpoint we are going to build an API to leverage the power of GraphQL interfaces.
We are going to use this dummy Rest API – Json Placeholder:

Request:

Response:

These GraphQL Types describe our data:

Let’s see how to create a new AppSync API.
Fist of all, create a new AWS AppSync client and then create a new API:

When you create a new API you need to specify the API name and the API authentication type. In the example I used the API_KEY authentication type. Here you can read more about authentication types: AWS AppSync Security .

Create a new API key:

Create new GraphQL types:

Data sources and resolvers are how AWS AppSync translates GraphQL requests and fetches information from your AWS resources.
AWS AppSync has support for automatic provisioning and connections with certain data source types. You can use a GraphQL API with your existing AWS resources or build data sources and resolvers. This section takes you through this process in a series of tutorials for better understanding how the details work and tuning options.

Create a new HTTP data source based on our API:

The next step is to create the HTTP Resolver.
A resolver uses a request mapping template to convert a GraphQL expression into a format that a data source can understand. Mapping templates are written in Apache Velocity Template Language (VTL).

This is how our request mapping template looks like.

This is how our response mapping template looks like.

Create the new resolver:

And we are done! We created an API that use REST APIs with AWS AppSync to leverage the power of GraphQL interfaces.


cURL request:

Response:

Extract business insights from audio using AWS Transcribe, AWS Comprehend and Elasticsearch – Part 1 and 2- Skedler Blog

I’ve just published a new blog post on the Skedler Blog.
In this two-part blog post, we are going to present a system architecture to convert audio and voice into written text with AWS Transcribe, extract useful information for quick understanding of content with AWS Comprehend, index this information in Elasticsearch 6.2 for fast search and visualize the data with Kibana 6.2. In Part I, you can learn about the key components, architecture, and common use cases. In Part II, you can learn how to implement this architecture.

The components that we are going to use are the following:

  • AWS S3 bucket
  • AWS Transcribe
  • AWS Comprehend
  • Elasticsearch 6.2
  • Kibana 6.2
  • Skedler Reports and Alerts

System architecture:

You can read the full post – Part 1 – here: Extract business insights from audio using AWS Transcribe, AWS Comprehend and Elasticsearch – Part 1.

Part 2 – here: Extract business insights from audio using AWS Transcribe, AWS Comprehend and Elasticsearch – Part 1.

Please share the post and let me know your feedbacks.

AWS Transcribe – Use cases

At the re:invent2017 AWS presented AWS Transcribe (read more about new AWS announcements here: AWS Comprehend, Translate and Transcribe).

Amazon Transcribe is an automatic speech recognition (ASR) service that makes it easy for developers to add speech to text capability to their applications. Using the Amazon Transcribe API, you can analyze audio files stored in Amazon S3 and have the service return a text file of the transcribed speech.

Use cases

Amazon Transcribe can be used for lots of common applications, for example:

  • transcription of customer service calls
  • generation of subtitles on audio and video content
  • conversion of audio file (for example podcast) to text
  • search for key words or not safe for work words within an audio file

The service is still in preview, watch the launch video here: AWS re:Invent 2017: Introducing Amazon Transcribe
You can read more about it here: Amazon Transcribe – Accurate Speech To Text At Scale.

Here how to use AWS Transcribe with the Python SDK. Please notice that the service is still in preview.
Here you can find the documentation: Boto3 AWS Transcribe.

Initialize the client:

Run the transcribe job:

Check the job status and print the result:

Here how the output looks like:

Soon I am going to use AWS transcribe to build a speech to text recognition system that will process customer care recordings in order to:

  • convert speech to text (using AWS Transcribe)
  • extract entites, sentiment and key phrases from the text (using AWS Comprehned)
  • index the result to Elasticsearch for fast search

How to combine Text Analytics and Search using AWS Comprehend and Elasticsearch 6.0 – Skedler Blog

I published a new blog post on the Skedler Blog.
In the post we are going to see how to combine text analytics and search using AWS Comprehend and Elasticsearch 6.0.

The components that we are going to use are the following:

  • Amazon S3 bucket and Amazon Simple Queue Service
  • Amazon Comprehend
  • Elasticsearch 6.0
  • Elasticsearch Ingest Attachment Processor Plugin
  • Elasticsearch Ingest Geoip Processor Plugin
  • Kibana
  • Skedler Reports and Alerts

System Architecture:


 
You can read the full post here: How to combine Text Analytics and Search using AWS Comprehend and Elasticsearch 6.0

Please share the post and let me know your feedbacks.

AWS Comprehend, Translate and Transcribe

At the re:invent2017 AWS presented a lot of new services (read all the announcements here: re:Invent 2017 Product Announcements). In this post we are going to see three new services related to the language processing.

  • Amazon Comprehend
  • Amazon Translate
  • Amazon Transcribe

These new services are listed within the Machine Learning section.

Amazon Comprehend

Amazon Comprehend is a natural language processing (NLP) service that uses machine learning to find insights and relationships in text. Amazon Comprehend identifies the language of the text; extracts key phrases, places, people, brands, or events; understands how positive or negative the text is; and automatically organizes a collection of text files by topic.

You can use the Amazon Comprehend APIs to analyze text and use the results in a wide range of applications including voice of customer analysis, intelligent document search, and content personalization for web applications.

The service constantly learns and improves from a variety of information sources, including Amazon.com product descriptions and consumer reviews – one of the largest natural language data sets in the world – to keep pace with the evolution of language.

You can read more about it here: AWS Comprehend and here Amazon Comprehend – Continuously Trained Natural Language Processing.
Watch the video from the AWS re:Invent Launchpad: Amazon Comprehend.

This service is available, here you can find few examples (using Boto3 Python AWS SDK).

Instantiate a new client.

Detect the dominant language in your text.

Detect the entities in your text.

Detect the key phrases in your text.

Get the sentiment in your text.

Amazon Translate

Amazon Translate is a neural machine translation service that delivers fast, high-quality, and affordable language translation. Neural machine translation is a form of language translation automation that uses machine learning and deep learning models to deliver more accurate and more natural sounding translation than traditional statistical and rule-based translation algorithms. Amazon Translate allows you to easily translate large volumes of text efficiently, and to localize websites and applications for international users.

The service is still in preview, watch the launch video here: AWS re:Invent 2017: Introducing Amazon Translate

You can read more about it here: Introducing Amazon Translate – Real-time Language Translation.

Amazon Transcribe

Amazon Transcribe is an automatic speech recognition (ASR) service that makes it easy for developers to add speech to text capability to their applications. Using the Amazon Transcribe API, you can analyze audio files stored in Amazon S3 and have the service return a text file of the transcribed speech.

Amazon Transcribe can be used for lots of common applications, including the transcription of customer service calls and generating subtitles on audio and video content. The service can transcribe audio files stored in common formats, like WAV and MP3, with time stamps for every word so you can easily locate the audio in the original source by searching for the text. Amazon Transcribe is continually learning and improving to keep pace with the evolution of language.

The service is still in preview, watch the launch video here: AWS re:Invent 2017: Introducing Amazon Transcribe
You can read more about it here: Amazon Transcribe – Accurate Speech To Text At Scale.

This is an example of how to use this service (code written by @jrhunt and taken from here).
Note that the API for Transcribe (while in preview) is subject to change (this code may not be the final version of the API):

Output of the speech recognition:

I am looking forward to use these new services, they are easy to use, they easily integrate with the AWS world and they can add powerful features to your applications.