Connect with us

FACEBOOK

How well is your bot doing?

Published

on

Tools to measure your Messenger bot analytics & performance

Measuring the performance of a bot is critical to understand what is and is not working, and how you can best serve your customers and achieve your business goals. This post reviews the top ways to measure the performance of your bot, like how many new users have interacted with your bot, or the number of blocked conversations your bot has received, and more. These performance indicators will enable you to grow your audience and provide a good feedback loop.

In this post we deep-dive on all the insights and metrics you can gather from your Page, Facebook Analytics, rating, and reviews once your bot is live and in production.

The Insights > Messages tab of the Page attached to your Facebook App gives a brief overview of messaging interactions people have had with your bot. This is a good spot to check your spam and block rates. Keep them low by focusing on a good user experience! You don’t need to do any set up to access these insights: once your bot is live and your page has had more than 100 likes, you’ll access the following five groups of metrics:

  • Active conversations: The number of conversations between your Page and people on Messenger.
  • Your responsiveness: The percentage of messages that you’ve answered and your average response time.
  • Deleted conversations: The number of conversations with your Page that people deleted.
  • Marked as spam: The number of conversations from your Page that were marked as spam.
  • Blocked conversations: The number and percentage of conversations from your Page that have been blocked.

Exporting the data

App admins can download this data as a CSV for analysis from the App Dashboard > Messenger > App Analytics. You can also use the Messaging Insights API to programmatically retrieve this information.

Dashboard example

Below is an example of what an analytics dashboard might look like. As you might expect, the numbers with green percentage rates are positive, and those with red need improvement. The steps you should take to improve the metric are typically indicated by the metric. For example, if your responsiveness is low, then initiate a new response policy by bringing more people onto your support team, or automating responses to tell people when to expect a follow up reply.

Access Page Insights for deeper analysis with 3rd party tools

If you are interested in deeper analysis, you can pull this data into a 3rd party or custom data analysis / storage tool. You can also access the same metrics found in page insights from the /insights API endpoint.

You can select the metrics you’re interested in by adding them as a comma separated list to the metric parameter. For example, to return page_total_messaging_connections use the following request:

curl -i -X GET "https://graph.facebook.com/v3.0/<PAGE_ID>/insights/?metric=page_total_messaging_connections&access_token=<PAGE_ACCESS_TOKEN>"

Which returns data like the following:

"data": [
{
"name": "page_total_messaging_connections",
"period": "day",
"values": [
{
"value": "200",
"end_time": "<UTC_TIMESTAMP>"
}
}
]

To limit the data returned, add since and until parameters with Unix timestamps.

curl -i -X GET "https://graph.facebook.com/v3.0/me/insights/?metric=page_messages_reported_conversations_by_report_type_unique&since=1521072000&until=1523880268&access_token=<PAGE_ACCESS_TOKEN>"

Facebook Analytics is a powerful tool for developers to understand their audiences better. It provides detailed aggregated and anonymized demographic data that users have opted to share, plus actions and events they took with your bot. These include pre-defined and custom events you can add to track user workflows relevant to your bot. Pre-defined events include events such as:

  • Someone adding (installing) your bot.
  • Conversation deleted.
  • A message sent to the bot.

There are a lot of sections, which varies based on how you have set your bot up, but let’s step through a couple of the most valuable.

Overview and Dashboards

When you first open Facebook Analytics, you will see an overview of data that you can filter by dates and segments. To create your own summaries of data, add a ‘Dashboard’ under the Dashboards menu item. From the new Dashboard, create a Dashboard with events, segments, and a date range to represent the metric. You can also use the Dashboard to display goals based on metrics and how close you are to meeting them.

For example, the Dashboard below shows a chart summarizing the activity of all female users of a bot.

Segments

Think of segments as queries of your data that allow you to filter by a series of properties based on what the analytics platform has already collected. For example, the segment below filters the data of European countries (collected so far) to show you data from the date range that would help you figure out if it’s worth targeting new regions with localized offers.

Activity

The activity sub-section contains a series of pre-defined summaries of user demographics and events, all of which you can filter with segments, time ranges, and other values relevant to each summary. You can also pin charts under each sub-section to dashboards.

Some of these sub-sections require further configuration to suit your use cases:

Funnels define a series of steps, and timelines between those steps, you would like users to follow. Clicking the funnel shows a summary of your current success rates in that scenarios. The image below shows the conversion rate and average amount of time it took from installing a bot to making a purchase.

Cohorts track the behavior of demographic groups over time and the patterns between them.

For example, the image below shows how long users take their second action after their first. For example, you can track how long it takes from people between signing up with your bot and starting a purchase process.

You can use breakdowns to create pivot tables of data based on up to three parameters. For example, the ages of users and what platform they install the Messenger bot in, again useful to know if it’s worth creating targeted offers.

Custom events

The analytics pre-defined events cover many eventualities, and depending on your business model and App setup, you might need custom events to track specific actions. Creating a custom event depends on the platform you are targeting, but for example, here’s how you create a custom event for bots running on the web that generates a custom event when a user clicks a custom button in a webview.

function onCustomButtonClick() {
FB.AppEvents.logEvent("custom_button_click");
}

You can also trigger pre-defined events in your code, and pass parameters with certain events. For example, to trigger an ADDED_TO_CART event:

function logAddedToCartEvent(contentData, contentId, contentType, currency, price) {
var params = {};
params[FB.AppEvents.ParameterNames.CONTENT] = contentData;
params[FB.AppEvents.ParameterNames.CONTENT_ID] = contentId;
params[FB.AppEvents.ParameterNames.CONTENT_TYPE] = contentType;
params[FB.AppEvents.ParameterNames.CURRENCY] = currency;
FB.AppEvents.logEvent(FB.AppEvents.EventNames.ADDED_TO_CART, price, params);
}

People

The people sub-section provides a summary of aggregated and anonymized demographics for the people using your bot and the devices they use.

Push campaigns

This subsection shows the performance of any current or historical push notification campaigns.

Facebook Platform

This sub-section summarizes how people using your bot have interacted with other aspects of the Facebook Platform. Relevant to Messenger bots, this includes referrals. Referrals are the number of users who found your bot from shared links and campaigns on Facebook.

Reviews of your bot (or page) provide you with a broad view of people’s opinion on your business or brand. People can leave ratings of up to 5 stars and free form feedback.

You can find the feedback and ratings left for your bot under the Products > Messenger > Bot Ratings tab of your bot’s dashboard. You can find the same data in the Messenger Reviews tab of the Page settings attached to the bot, and they are emailed to you daily.

You can’t reply to these reviews, but as they are one of the first metrics people see when deciding whether to use your bot, you should use them for overall guidance on how to improve your bot experience and encourage happy customers to add positive reviews.

There are many ways you can track, analyze, and interact with your bot’s data. From custom events and segments to combining your Facebook Analytics with other sources of data to make high-level, data-informed decisions. Monitoring the performance of you bot helps you iterate and improve not only the infrastructure and UX of your bot, but also your audience’s experience.

Messenger Developer Blog

FACEBOOK

Preparing our Partners for iOS 14: Actions for Partners and Mobile Web Advertisers

Published

on

Today, January 19, we are providing an update to our earlier guidance to prepare you for the changes created by Apple’s iOS 14 requirements. We’re launching new experiences in our advertising tools that will guide you through some of these changes, including some of the actions you will soon be able to take. While Apple has still not made it clear to the industry when they will enforce their AppTransparencyTracking prompt, we are taking advance steps to help prepare you and reduce disruption to your advertising and integrations across Facebook apps.

Please note if you use the Ads Insights API, there will be an immediate change to measurement beginning today. Please see details below.

RESOURCE CENTER

We will launch Resource Center, a dedicated tab in Ads Manager with a customized checklist of tasks to guide advertisers through specific actions to ensure your advertising is set up optimally and prepare for the upcoming impact of iOS 14 requirements.

The Resource Center will provide a list of relevant updates summarizing how iOS 14 requirements may impact your specific ad account. Most tasks and updates will link to Help Center articles to further support you through these changes.

EVENT MANAGEMENT

Beginning today, advertisers will be able to view all their events through new tabs in Events Manager, “Pixel/Conversions API” and “Aggregated Event Measurement.” The Aggregated Event Measurement tab will provide information on web events processed using Aggregated Event Measurement. Pixel/Conversions API tab Full data events includes all other web events that are not limited by Aggregated Event Measurement. Initially, the Aggregated Event Measurement tab will include events autoconfigured for the accounts domains based on an accounts ad activity.

Advertisers will be able to edit and prioritize the automatically configured events for a given domain in a new event configuration tool in Events Manager.

Below you will find previous guidance and additional details provided for Aggregated Event Measurement and Event Limits.

Aggregated Event Measurement: Facebook will introduce Aggregated Event Measurement to support measurement of web events from iOS 14 users once Apple requires the ATT prompt. It is designed to help you measure campaign performance in a way that is consistent with consumers’ decisions about their data.

Event Limits: You will be limited to the use of 8 conversion events per domain (i.e., 8 <pixel, event> or <custom conversion> per domain) for campaign optimization. You will not need to make changes to the pixel or your Conversions API implementation as event selection will be done in Events Manager.

  • Ahead of the prompt enforcement, advertisers will be able to view, edit and prioritize the 8 conversion events for a given domain.
  • Prior to enforcement, configuring the 8 conversion events for a domain will not impact optimization or reporting
  • At the time of enforcement, ad sets optimizing for a conversion event that is no longer available will be paused.
  • The 8 conversion events per domain will be ranked based on priority. If multiple events are completed by a user (i.e. “add to cart” and “purchase”) only the higher prioritized event will be reported.

Actions to take now:

Domain verification in Business Manager: We recommend that all advertisers verify their domain using the steps outlined in the Facebook Help Center. However, if a domain has pixel events owned by multiple Business Managers or ad accounts, one Business Manager is required to verify the domain in order to edit the event configuration.

Note: Domain Verification must be done at the effective top level domain plus one (eTLD+1 ). For example, for www.books.jasper.com, books.jasper.com and jasper.com the eTLD+1 domain is jasper.com.

Plan for 8 conversion events per domain: If you use more than 8 conversion events per domain for optimization prepare an action plan for how to operate with 8 conversion events per domain based on your business priorities. You will not need to make changes to the pixel or Conversions API implementation as event configuration will be done in Events Manager.

DELIVERY

VALUE OPTIMIZATION (VO)

  • VO will transition from Ads Manager to Events Manager and value set will need to be enabled.
  • If you have previously used VO, value sets will be automatically assigned based on historical data and turned on in Events Manager.
  • If you are eligible and use VO infrequently or have never used it, value sets must be configured manually.
  • You will be able to have a maximum of 8 value sets. Note: turning on value sets will automatically utilize 4 out of the 8 events allowed for campaign optimization and reporting for a given domain.
  • For optimal performance, events with value sets turned on should be placed in higher priority slots within the Events Manager.

Actions to take now:

DYNAMIC ADS

There are no specific changes for Dynamic Ads for Retargeting, but you may see performance and audience sizes decrease. We expect minimal impact to you using Dynamic Ads to reach Broad Audiences.

Actions to take now:

  • Verify product URL domains in the catalog feed and avoid the use of URLs redirecting users to a different domain.
  • Prepare to use only 1 pixel per catalog to optimize for prioritized conversion events across all catalog items.

MEASUREMENT

Ads Reporting (Ads Manager, Ads Reporting, Ads Insights API):

The changes outlined below will be going into effect January 19, 2021. Visit our business help center for more information.

  • We will be replacing the account level attribution window with a new attribution setting at the ad set level accessed during campaign creation and ensures that the conversions we measure are the same ones that inform campaign optimization, allowing for increased clarity when analyzing ad performance. This setting will also be available to query via the Ads Insights API. Please reference the recent Ads Insights API blog post for more information on two new fields related to attribution settings that can be used: use_unified_attribution_setting and attribution_setting.
  • To prepare our advertisers for upcoming changes resulting from Apple’s enforcement of their AppTrackingTransparency (ATT) prompt, beginning January 19, 2021, we will initially default to a 7-day click and 1-day view attribution setting. Due to limitations around view-through signal, the attribution setting for newly created campaigns after prompt enforcement will default to 7-day click instead of 7-day click and 1-day view.
  • Any active campaigns along with future campaigns will report and optimize on the new attribution setting. While these changes don’t impact ad delivery and we will not change the optimization window for any active campaigns, the new default attribution setting may result in a decrease in the number of reported conversions, especially for advertisers who have currently opted for reporting based on longer attribution windows like 28-day click and 1-day view.
  • Until Apple begins to enforce their App Tracking Transparency (ATT) prompt, you can continue to access data for all existing attribution windows (such as 28-day click, 28-day view, 7-day view) with the comparing windows feature. Once Apple enforces their ATT prompt, we will no longer support 28-day click, 28-day view and 7-day view windows and historical data for deprecated attribution windows will only be accessible via the Ads Insights API.
  • Statistical modeling will be used for certain attribution windows and/or metrics to account for less data availability from iOS 14 users. In-product annotation will communicate when a metric is modeled.
  • Certain attribution windows will have partial reporting and metrics will not include all events from iOS 14 users. In-product annotation will communicate when a metric is partial. This will launch in early 2021 upon Apple’s enforcement of their ATT prompt.
  • Delivery and action breakdowns will not be supported for offsite conversion events.
  • Offsite conversion events will be reported based on the time the conversions occur and not the time of ad impressions. As a result, you may notice small fluctuations in cost metrics, as cost per conversion will reflect spend over a given period divided by conversions that took place over the same period, whereas today, cost per conversion reflects spend over a given period divided by conversions driven by ad impressions that took place over the same period.

For automated rules: In early 2021, attribution window settings for existing and newly created automated rules will no longer be supported, and a default window of 7-day click-through for non-iOS conversions and the SKAdNetwork window for iOS conversions will be implemented.

Actions to take now Specify attribution window changes (deprecation of 28-day click-through, 28-day view-through, and 7-day view-through windows):

  • Leverage the Comparing Windows feature to see how conversions attributed to ads compare across different attribution windows. This allows you to better anticipate the impact to reported conversions as a result of upcoming attribution window changes.
  • Update any automated rules currently using a 28-day attribution window to prevent any unexpected adjustments in spend once the new 7-day click-through window default goes into effect.

DEVELOPER APIs (Marketing API, Ads Insights API, Targeting API)

Please review the most recent guidance published here.

For information on iOS 14 impact to App based developers please see our separate Developer blog post here.

For information on how to plan for changes to our Marketing & Ads Insights API, please see our blog post here.

Facebook Developers

Continue Reading

FACEBOOK

Preparing our Partners for iOS 14: Actions for App Advertisers and Developers

Published

on

Today, January 19, we are providing an update to our earlier guidance to prepare you for the changes created by Apple’s iOS 14 requirements. We’re launching new experiences in our advertising tools that will guide you through some of these changes, including actions you will soon be able to take. While Apple has still not made it clear to the industry when they will enforce their AppTransparencyTracking prompt, we are taking advance steps to help prepare you and reduce disruption to your advertising and integrations across Facebook apps. Below you will find the following updates:

  • New experiences in Events Manager, including a dedicated resource center
  • Updates to Ads Measurement
  • New Limited Login Mode for Facebook Login

Please note if you use the Marketing API or Ads Insights API please review this blog in detail for changes that will impact your campaigns. If you use the Ads Insights API there will be an immediate change to measurement beginning today.

As a review, advertisers using our Facebook SDK, Audience Network SDK, or and/or a Mobile Measurement Partner SDK need to be aware of the following version requirements:

FACEBOOK SDK

  • Advertisers who have already upgraded to Facebook SDK v8.1 or above will be able to configure their SKAdNetwork for App install ads via standard or customizable methods.
  • Advertisers who have not already upgraded should upgrade to Facebook SDK v9.0 which was released today.

AUDIENCE NETWORK SDK

  • All publishers will need to use Audience Network SDK v6.2.1 in order to monetize with iOS 14 users when Apple introduces their new requirements. Audience Network SDK v6.2.1 was released on January 11, 2021.

FOR ADVERTISERS WHO WORK WITH MOBILE MEASUREMENT PARTNERS (MMPs)

  • If you work with an MMP and do not use the Facebook SDK, check with your MMP for more information regarding their completion of integrating with Facebook to support SKAdNetwork. If completed, the MMP will be able to provide you with a unique URL to paste into Events Manager to be able to run AEO and VO campaigns.
  • If you work with an MMP and use the Facebook SDK:
    • When optimizing for Installs (MAI): App should have the latest version of the Facebook SDK (v8.1 or above) or equivalent version of MMP SDK. This requires MMPs to have completed integration with Facebook to support interoperability of SKAdNetwork. Please check with your respective MMP for more information on where they stand on completion of integration with Facebook to support MAI using SKAdNetwork.
    • For optimizing app events, value, or mobile app installs + events, you will need to select one method to set measurement goals/conversion bits:
      • If events are sent from Facebook SDK, use Events Manager to set configure conversion schema.
      • If events are sent from MMP, the MMP interface needs to be used to configure the conversion schema.

EVENTS MANAGER:

  • Starting January 19, 2021, ahead of Apple’s prompt enforcement, advertisers will be able to configure the SKAdNetwork conversion schema in Events Manager to measure and optimize for App Events (AEO), Value (VO), or App Installs + App Events (MAI+events).

APP INSTALL ADS:

  • Starting January 19, 2021, ahead of Apple’s prompt enforcement, we will begin rolling out the ability for advertisers to set up and test SKAdNetwork-based app install campaigns in Ads Manager and via API.
  • Starting January 19, 2021, advertisers will see a toggle at the campaign level to indicate that they want to run an app install campaign targeting iOS 14 users.

ADS MEASUREMENT

  • We will be replacing the account level attribution window with a new attribution setting accessible during campaign creation at the ad set level. For active and new ad sets, the setting will default to 7-day click and 1-day view, which may result in a decrease in the number of reported conversions. Due to limitations around view-through signal, the attribution setting for newly created campaigns after prompt enforcement will default to 7-day click instead of 7-day click and 1-day view. Today, you can prepare for attribution window changes by using the Comparing Windows feature in Ads Manager to see performance across all existing windows.

APP EVENTS API

  • If you use App Events API and do not use Facebook SDK to support SKAdNetwork API, confirm that you have properly implemented SKAdNetwork through Events Manager to access the SKAdNetwork configuration experience.
  • Once confirmed, you will be able to configure your conversion schema to run AEO and VO campaigns

RESOURCE CENTER

We are launching Resource Center, a dedicated tab in Ads Manager with a customized checklist of tasks to guide you through actions you can take to ensure your advertising is set up optimally and prepare for the upcoming impact of iOS 14 requirements. The Resource Center will provide a list of relevant updates summarizing how iOS 14 requirements may impact your specific ad account. Most tasks and updates will link to Help Center articles to further support you through these changes.

FACEBOOK LOGIN:

  • Facebook Login will now have a Limited Login mode that implements safeguards that allow advertisers to choose whether Facebook can use the fact that a person used Facebook to log into an advertiser’s iOS app to target advertising or measure advertising effectiveness. To implement this new version of Facebook Login, advertisers must update to version 9.0+ of the Facebook iOS SDK or Facebook SDK for Unity.
  • For more information on this new version of Facebook Login, including implementation documentation and FAQs, we encourage you to visit our Developers page.

Facebook Developers

Continue Reading

FACEBOOK

Introducing Facebook Platform SDK Version 9.0

Published

on

Today, we are releasing Facebook SDK version 9.0 and additional updates to our Facebook Platform SDKs. These updates include additional features, as well as required actions which may impact your application(s) integration with our platform. This post outlines these updates and the necessary steps developers need to take to avoid disruption where applicable.

Along with the release of Facebook SDK version 9.0, we are announcing the deprecation of all SDK versions below version 9.0. Beginning today, developers will need to begin migrating to version 9.0 to prevent usage of deprecated SDKs for their associated application(s).

Detailed information regarding the deprecation and sunset policy is provided below.

Release of Facebook Platform SDK Version 9.0 and Updates

Facebook Login Updates: A New Limited Data Mode

Facebook Login now offers a Limited Login mode that implements safeguards designed to prevent the fact that a person used Facebook to log into an iOS app from being used to target advertising or measure advertising effectiveness. To implement this new version of Facebook Login, Developers should update their Facebook iOS SDK or Facebook SDK for Unity to version 9.0+.

Learn more about the changes to Facebook Login here

Deprecation of Facebook Platform SDKs Below Version 9.0

Deprecations

Today, we will launch Facebook Platform SDK version 9.0 and begin the deprecation of all prior SDK versions. The deprecation will happen over a two year period (ending on January 19th, 2023) at the end of which all previous versions of the Facebook Platform SDK will be permanently sunset. At that point, no responses will be generated for any API calls made to previous versions (v8.2 and below) of the Facebook Platform SDK. Through this process, the Facebook Platform SDK versioning will align with the Graph API versioning commitment.

As we continue to improve our platform and products, we encourage all developers to adopt our newest version of the SDK. Version 9.0 includes improvements to crash rate prevention and tracking, error testing, memory usage and much more.

We are committed to our SDKs and will continue to make improvements that help developers work with our platform. When developers upgrade to version 9.0, this allows us to focus on the stability of supported SDK versions and improve the developer experience by mitigating potential privacy, stability, compatibility and security concerns that stem from older versions.

New Versioning Schedule for Facebook Platform SDK Major Releases

Moving forward, all new major versions will target annual releases with the goal of aligning the SDK version number to match the latest Graph API version. We will continue to release minor updates to enhance the functionality of our Facebook SDKs and we may release major versions off-cycle if needed.

End of Official Support for Facebook SDK for React Native

Today, Facebook will end official support for our React Native wrapper around the Facebook SDKs for Android and iOS. We are pleased by the community’s efforts that make the Facebook SDK for React Native a success. We believe the community is well equipped to address developer needs going forward. Note that our support for React Native continues and is not affected by this.

The current version of the project will move to Facebook Archive. We recommend the community fork this repo into a new project that can be continuously maintained by the community. We encourage the community to make any necessary changes that they believe will enhance the functionality of the SDK moving forward.

Removal of Auto Initialization of Read Library

Currently, the Facebook Platform SDKs automatically initializes on launch. We will be removing this auto-initialization feature beginning today.

If you currently rely on the Facebook Platform SDKs being automatically initialized in order to get information about your application’s usage, you will now need to initialize the SDKs prior to gathering AppEvents data. More information about initializing the SDKs can be found at – https://developers.facebook.com/docs/

Visit our Change Logs to review specific changes to your SDK and stay informed on planned improvements:

Version Deprecations:

Below are the versions of the Facebook Platform SDKs that are deprecated and associated dates:

  • January 19, 2021 – iOS SDK v8.2 and below
  • January 19, 2021 – Android SDK v8.2 and below
  • January 19, 2021 – Unity SDK v8.2 and below

Facebook Developers

Continue Reading

Trending