Connect with us

FACEBOOK

How well is your bot doing?

Published

on

Tools to measure your Messenger bot analytics & performance

Measuring the performance of a bot is critical to understand what is and is not working, and how you can best serve your customers and achieve your business goals. This post reviews the top ways to measure the performance of your bot, like how many new users have interacted with your bot, or the number of blocked conversations your bot has received, and more. These performance indicators will enable you to grow your audience and provide a good feedback loop.

In this post we deep-dive on all the insights and metrics you can gather from your Page, Facebook Analytics, rating, and reviews once your bot is live and in production.

The Insights > Messages tab of the Page attached to your Facebook App gives a brief overview of messaging interactions people have had with your bot. This is a good spot to check your spam and block rates. Keep them low by focusing on a good user experience! You don’t need to do any set up to access these insights: once your bot is live and your page has had more than 100 likes, you’ll access the following five groups of metrics:

  • Active conversations: The number of conversations between your Page and people on Messenger.
  • Your responsiveness: The percentage of messages that you’ve answered and your average response time.
  • Deleted conversations: The number of conversations with your Page that people deleted.
  • Marked as spam: The number of conversations from your Page that were marked as spam.
  • Blocked conversations: The number and percentage of conversations from your Page that have been blocked.

Exporting the data

App admins can download this data as a CSV for analysis from the App Dashboard > Messenger > App Analytics. You can also use the Messaging Insights API to programmatically retrieve this information.

Dashboard example

Below is an example of what an analytics dashboard might look like. As you might expect, the numbers with green percentage rates are positive, and those with red need improvement. The steps you should take to improve the metric are typically indicated by the metric. For example, if your responsiveness is low, then initiate a new response policy by bringing more people onto your support team, or automating responses to tell people when to expect a follow up reply.

Access Page Insights for deeper analysis with 3rd party tools

If you are interested in deeper analysis, you can pull this data into a 3rd party or custom data analysis / storage tool. You can also access the same metrics found in page insights from the /insights API endpoint.

You can select the metrics you’re interested in by adding them as a comma separated list to the metric parameter. For example, to return page_total_messaging_connections use the following request:

curl -i -X GET "https://graph.facebook.com/v3.0/<PAGE_ID>/insights/?metric=page_total_messaging_connections&access_token=<PAGE_ACCESS_TOKEN>"

Which returns data like the following:

"data": [
{
"name": "page_total_messaging_connections",
"period": "day",
"values": [
{
"value": "200",
"end_time": "<UTC_TIMESTAMP>"
}
}
]

To limit the data returned, add since and until parameters with Unix timestamps.

curl -i -X GET "https://graph.facebook.com/v3.0/me/insights/?metric=page_messages_reported_conversations_by_report_type_unique&since=1521072000&until=1523880268&access_token=<PAGE_ACCESS_TOKEN>"

Facebook Analytics is a powerful tool for developers to understand their audiences better. It provides detailed aggregated and anonymized demographic data that users have opted to share, plus actions and events they took with your bot. These include pre-defined and custom events you can add to track user workflows relevant to your bot. Pre-defined events include events such as:

  • Someone adding (installing) your bot.
  • Conversation deleted.
  • A message sent to the bot.

There are a lot of sections, which varies based on how you have set your bot up, but let’s step through a couple of the most valuable.

Overview and Dashboards

When you first open Facebook Analytics, you will see an overview of data that you can filter by dates and segments. To create your own summaries of data, add a ‘Dashboard’ under the Dashboards menu item. From the new Dashboard, create a Dashboard with events, segments, and a date range to represent the metric. You can also use the Dashboard to display goals based on metrics and how close you are to meeting them.

For example, the Dashboard below shows a chart summarizing the activity of all female users of a bot.

Segments

Think of segments as queries of your data that allow you to filter by a series of properties based on what the analytics platform has already collected. For example, the segment below filters the data of European countries (collected so far) to show you data from the date range that would help you figure out if it’s worth targeting new regions with localized offers.

Activity

The activity sub-section contains a series of pre-defined summaries of user demographics and events, all of which you can filter with segments, time ranges, and other values relevant to each summary. You can also pin charts under each sub-section to dashboards.

Some of these sub-sections require further configuration to suit your use cases:

Funnels define a series of steps, and timelines between those steps, you would like users to follow. Clicking the funnel shows a summary of your current success rates in that scenarios. The image below shows the conversion rate and average amount of time it took from installing a bot to making a purchase.

Cohorts track the behavior of demographic groups over time and the patterns between them.

For example, the image below shows how long users take their second action after their first. For example, you can track how long it takes from people between signing up with your bot and starting a purchase process.

You can use breakdowns to create pivot tables of data based on up to three parameters. For example, the ages of users and what platform they install the Messenger bot in, again useful to know if it’s worth creating targeted offers.

Custom events

The analytics pre-defined events cover many eventualities, and depending on your business model and App setup, you might need custom events to track specific actions. Creating a custom event depends on the platform you are targeting, but for example, here’s how you create a custom event for bots running on the web that generates a custom event when a user clicks a custom button in a webview.

function onCustomButtonClick() {
FB.AppEvents.logEvent("custom_button_click");
}

You can also trigger pre-defined events in your code, and pass parameters with certain events. For example, to trigger an ADDED_TO_CART event:

function logAddedToCartEvent(contentData, contentId, contentType, currency, price) {
var params = {};
params[FB.AppEvents.ParameterNames.CONTENT] = contentData;
params[FB.AppEvents.ParameterNames.CONTENT_ID] = contentId;
params[FB.AppEvents.ParameterNames.CONTENT_TYPE] = contentType;
params[FB.AppEvents.ParameterNames.CURRENCY] = currency;
FB.AppEvents.logEvent(FB.AppEvents.EventNames.ADDED_TO_CART, price, params);
}

People

The people sub-section provides a summary of aggregated and anonymized demographics for the people using your bot and the devices they use.

Push campaigns

This subsection shows the performance of any current or historical push notification campaigns.

Facebook Platform

This sub-section summarizes how people using your bot have interacted with other aspects of the Facebook Platform. Relevant to Messenger bots, this includes referrals. Referrals are the number of users who found your bot from shared links and campaigns on Facebook.

Reviews of your bot (or page) provide you with a broad view of people’s opinion on your business or brand. People can leave ratings of up to 5 stars and free form feedback.

You can find the feedback and ratings left for your bot under the Products > Messenger > Bot Ratings tab of your bot’s dashboard. You can find the same data in the Messenger Reviews tab of the Page settings attached to the bot, and they are emailed to you daily.

You can’t reply to these reviews, but as they are one of the first metrics people see when deciding whether to use your bot, you should use them for overall guidance on how to improve your bot experience and encourage happy customers to add positive reviews.

There are many ways you can track, analyze, and interact with your bot’s data. From custom events and segments to combining your Facebook Analytics with other sources of data to make high-level, data-informed decisions. Monitoring the performance of you bot helps you iterate and improve not only the infrastructure and UX of your bot, but also your audience’s experience.

Messenger Developer Blog

FACEBOOK

Messenger API to support Instagram

Published

on

Today, we are announcing updates to the Messenger API to support Instagram messaging, giving businesses new tools to manage their customer communications on Instagram at scale. The new API features enable businesses to integrate Instagram messaging with their preferred business applications and workflows; helping drive more meaningful conversations, increase customer satisfaction and grow sales. The updated API is currently in beta with a limited number of developer partners and businesses.

Instagram is a place for emerging culture and trend creation and discovering new brands is a valuable part of this experience. Messaging plays a central role in helping people connect with brands in personal ways through story replies, direct messages, and mentions. Over the last year, total daily conversations between people and businesses on Messenger and Instagram grew over 40 percent. For businesses, the opportunity to drive sales and improve customer satisfaction by having meaningful interactions with people on Instagram messaging is huge.

Instagram is a platform for community building, and we’ve long approached it as a way for us to connect with our customers in a place where they are already spending a lot of their time. With the newly launched Messenger API support for Instagram, we are now able to increase efficiency, drive even stronger user engagement, and easily maintain a two-way dialogue with our followers. This technology has helped us create a new pipeline for best-in-class service and allows for a direct line of communication that’s fast and easy for both customers and our internal team.” – Michael Kors Marketing

Works with your tools and workflows

Businesses want to use a single platform to respond to messages on multiple channels. The Messenger API now allows businesses to manage messages initiated by people throughout their Instagram presence, including Profile, Shops, and Stories. It will be possible for businesses to use information from core business systems right alongside Instagram messaging, enabling more personal conversations that drive better business outcomes. For example, businesses integrating with a CRM system can give agents a holistic view of customer loyalty. Furthermore, existing investments in people, tools, and workflows to manage other communication channels can be leveraged and extended to support customers on Instagram. This update will also bring Facebook Shops messaging features to the Messenger API so businesses can create more engaging and connected customer experiences.

To get started, businesses can easily work with developers to integrate Instagram messaging with their existing tools and systems.

Increases responsiveness and customer satisfaction

Customers value responsiveness when they have questions or need help from businesses. For the first time on Instagram, we’re introducing new features that will allow businesses to respond immediately to common inquiries using automation, while ensuring people are seamlessly connected to live support for more complex inquiries. One of our alpha partners, Clarabridge, reported their client brands had improved their response rate by up to 55% since being able to manage Instagram DMs through their platform.

The updates to the Messenger API are part of our overall effort to make it easier for businesses to reach their customers across our family of apps.

Messenger API support for Instagram is currently in beta with a focus on providing high quality, personalized messaging experiences on Instagram while increasing business efficiency. Adidas, Amaro, Glossier, H&M, MagazineLuiza, Michael Kors, Nars, Sephora and TechStyle Fashion Group and other consumer brands are already participating in the beta program. We are excited about early results some businesses saw during alpha testing, including higher response rates, reduced resolution times, and deeper customer insights as a result of integrations. We’re also testing with a limited number of developer partners; and are delighted at the initial response.

On average, brands have saved at least four hours per agent per week by streamlining social community management within the Khoros platform, plus shortened response rates during business hours — which is crucial to meet as customers who message brands on social media expect a quick reply.” – Khoros

If you are a business interested in utilizing the API post-beta, please sign up for our business waitlist. If you are a developer, please sign up for our developer waitlist.

Facebook Developers

Continue Reading

FACEBOOK

Required migration to token-based access for User Picture and oEmbed endpoints

Published

on

As part of our Graph API 8.0 release, we announced breaking changes to how developers can access certain permissions and APIs. Starting October 24, 2020, developers need to migrate to token-based access in order to access User Picture and oEmbed endpoints for Facebook and Instagram.

This post outlines these changes and the necessary steps developers need to take to avoid disruption to their app.

User Picture

Facebook will now require client or app access tokens to access a user’s profile picture. Beginning on October 24, 2020 queries for profile pictures made against user IDs without an access token will return a generic silhouette rather than a profile picture. This is a breaking change for partners. While client or app tokens will be required for user ID queries, they will continue to be a best practice (and not required) for ASID queries for the time being.

Facebook and Instagram oEmbed

We are also deprecating the existing Legacy API oEmbed endpoints for Facebook and Instagram on October 24, 2020, which will be replaced with new Graph API endpoints. If developers don’t make this change and continue to attempt to call the existing oEmbed API, their requests will fail and developers will receive an error message instead. These new endpoints will require client or app access tokens or ASID queries.

Ready to make the switch? You can read more about these changes in our developer documentation for User Picture and also visit our changelog for Facebook OEmbed and IG OEmbed for details on how to start calling these Graph API endpoints.

Facebook Developers

Continue Reading

FACEBOOK

PyTorch Tutorials Refresh – Behind the Scenes

Published

on

Hi, I’m Jessica, a Developer Advocate on the Facebook Open Source team. In this blog, I’ll take you behind the scenes to show you how Facebook supports and sustains our open source products – specifically PyTorch, an open source deep learning library. With every new release version, PyTorch pushes out new features, updates existing ones, and adds documentation and tutorials that cover how to implement these new changes.

On May 5, 2020, PyTorch released improvements to Tutorials homepage with new content and a fresh usability experience out into the world (see the Twitter thread) for the community. We introduced keyword based search tags and a new recipes format (bite-sized, ready-to-deploy examples) and more clearly highlighted helpful resources, which resulted in the fresh homepage style you see today.

Something Went Wrong

We’re having trouble playing this video.

As the framework grows with each release, we’re continuously collaborating with our community to not only create more learning content, but also make learning the content easier.

What Changed

The tutorials refresh project focused on re-envisioning the learning experience by updating the UX and updating the learning content itself.

Our 3 major goals for the refresh were:

  • Reduce blocks of text and make it easy for users to find important resources (e.g. PyTorch Cheat Sheet, New to PyTorch tutorials)
  • Improve discoverability of relevant tutorials and surface more information for users to know about the available tutorial content
  • Create content that allows users to quickly learn and deploy commonly used code snippets

And we addressed these goals by:

  • Adding callout blocks with direct links to highlight important resource such as the beginner tutorial, the PyTorch Cheat Sheet and new recipes
  • Adding filterable tags to help users easily find relevant tutorials; and formatting the tutorials cards with summaries so users know what to expect without having to click in
  • Creating a new learning format, Recipes, and 15 brand new recipes covering some of the most popular PyTorch topics such as interpretability and quantization as well as basics such as how to load data in PyTorch
  • In summary:

Goals

Implementation

Reduce blocks of text and make it easy for users to find important resources (e.g. PyTorch Cheat Sheet, New to PyTorch tutorials)

Add callouts with direct links to highlight important resources

Improve discoverability of relevant tutorials and surface more information for users to know about the available tutorial content

Add filterable tags to help users easily find relevant tutorials. Reformat tutorial cards with summaries so users know what to expect

Create content that allow users to quickly learn and deploy commonly used code snippets

Create a new learning format – Recipes. These are bite-sized, actionable examples of how to use specific Pytorch features, different from our previous full-length tutorials

Why We Made These Changes

Now, what drove these changes? These efforts were driven by feedback from the community; two sources of feedback were the UX research study and direct community interactions:

  • UX Research study – Earlier in 2020, we conducted a UX research study of our website in collaboration with the Facebook UX Research team to understand how our developer community is using the website and evaluate ways we can improve it to better meet their needs
  • In-person events and online feedback – The team gathered community feedback about existing tutorials to help identify learning gaps.

We used these channels of input to fuel revisioning our learning experience.

Rethinking the Learning Experience

Given the feedback from the UX Research study and the in-person workshop, we went back and rethought the current learning experience.

  • 3 levels
    PyTorch Tutorials Refresh - Behind the Scenes
    • Level 1: API docs. Currently these exist and they contain code snippets that provide an easily understandable (and reproducible) example that allows a user to implement that particular API
    • Level 2: The missing puzzle piece
    • Level 3: Tutorials ideally provide an end-to-end experience that provides users an understanding of how to take data, train a model and deploy it into a production setting using PyTorch. These exist, but needed to be pruned of outdated content and cleaned up to better fit this model
    • Realized we were missing something in between, content that was short, informative, and actionable. That’s how recipes were born. Level 2: Recipes are bite-sized, actionable examples of how to use specific PyTorch features, different from our tutorials

What Was the Process

Just as it took a large team effort, this was more of a marathon as opposed to a sprint. Let’s look at the process:

Timeline of the process:

PyTorch Tutorials Refresh - Behind the Scenes

Overall, the project took about 6 months, not including the UX research and prior feedback collection time. It started off with the kickoff discussion to align on the changes. We assessed the existing tutorials, pruned outdated content and decided on new recipe topics and assigned authors. In the meantime, marketing and documentation engineers collaborated with our web design team on the upcoming UI needs, created mocks to preview with the rest of the team and built out the infrastructure.

For logistics, we created a roadmap and set milestones for the team of authors. We held weekly standup meetings, and the team bounced ideas in chat. The changes were all made in a staging branch in GitHub, which allowed us to create previews of the final build. Next, the build process. Many of the recipe authors were first time content creators, so we held a live onboarding session where we discussed teaching mindset, writing with an active voice, outlining, code standards and requirements; and this was all captured in a new set of content creation documentation.

The bulk of the process was spent in building out the content, copy editing and implementing the UI experience.

With the product out the door, we took some time to perform a team retrospective – asking what went well? What went poorly? What can we do better next time? In addition, we continue to gather ongoing feedback from the community through GitHub issues.

Moving forward, we are brainstorming and forming a longer-term plan for the PyTorch learning experience as it relates to docs and tutorials.

Ways to Improve

Looking back on ways we could have improved:

  • Timeline – Our timeline ended up taking longer than anticipated because it had been coupled with a previous version release and team members were serving double-duty working on release content, as well as tutorials refresh content. As version release approached, we took a step back and realized we needed more time to put out a more polished product.
  • Testing – In software development, if there is an impending deadline, typically testing is the first thing to get squished; however, more focused testing will always save time in the bigger picture. For us, we would always welcome more time for more CI tests of the tutorial build, as well as beta tests of the user experience. Both of these are ongoing works in progress, as we continue to improve the tutorials experience overall.

What’s Next

So what’s next? We understand that this was just one change in a larger landscape of the overall PyTorch learning experience, but we are excited to keep improving this experience for you, our dedicated PyTorch user.

We would like to hear from you about your experience in the new tutorials. Found a tutorial you loved? Tweet about it and tag us (@PyTorch). Ran into an issue you can help fix? File an issue in https://github.com/pytorch/tutorials. We are excited to continue building the future of machine learning with you!

To learn more about Facebook Open Source, subscribe on YouTube, or follow us on Twitter and Facebook.

Facebook Developers

Continue Reading

Trending