For today’s interview, we have Daniel Xu who is a Production Engineer on the Linux userspace team at Facebook. The Linux userspace team works on a wide variety of systems tools, libraries, and services, all of which deal directly with the Linux kernel. While working on this team, Daniel has been using Rust as one of the main languages for development. Most of his work these days centers around observability, particularly dynamic tracers and debuggers. Let’s hear from him about how his experience with Rust has been and learn more about his work.
In what capacity have you been using Rust?
I’ve used Rust to write services, libraries and standalone tools over the last year and a half to solve low level problems such as making system calls, parsing binary data, making Foreign function interface (FFI) calls to C libraries, etc. These projects have high correctness requirements and any bugs would cause issues that would be very hard to debug or require specialized knowledge for the end user.
Why did you/your team at Facebook choose to use Rust over other languages?
I had heard all the fanfare surrounding Rust and was naturally hesitant – choosing a programming language for a non-trivial task is an important design choice. A wrong choice could lead to frustration, technical debt, or worse. Luckily for me, my coworker, who is a long-time systems expert and C++ programmer, was also a fan of Rust and encouraged me to try it out. On their advice, I took a leap of faith and started writing Below, an interactive tool to view and record historical system data, along with a few other enthusiastic colleagues.
On paper, Rust seemed like a natural choice. Below had to parse a lot of textual data, it had high uptime requirements, and it had to be efficient. These factors play into Rust’s strengths. Rust’s emphasis on memory safety by default and correctness-if-it-compiles allows us to move faster and be more confident that we’re writing correct and safe code. The deal is further sweetened by the fact that Rust compiles to native code and has a relatively lightweight runtime, when compared to Python/Go/Java – every bit of efficiency matters when Below is running on millions of hosts.
What are some of the projects that you’ve worked on at Facebook that use Rust?
Below is one of the major projects I’ve worked on in Rust. Below is a system monitor: think of it like an interactive process viewer such as top or htop, except that it records historical data to disk and is also cgroup aware. Engineers across the company rely on Below to debug system issues on any of the millions of the bare-metal hosts the company manages.
Another project I’ve written in Rust is libbpf-rs, which provides safe and idiomatic bindings to libbpf (a C library) that allow Rust developers to work with extended Berkeley Packet Filter (eBPF). This project was developed in the open and is currently used by a number of external developers. To learn more about libbpf, visit my blog, where I’ve written more extensively on this topic.
Two other fairly niche projects I’ve worked on are btrfs-fuzz and btrd. Btrfs-fuzz is an unsupervised, coverage-guided fuzzer for btrfs, a next-generation Linux filesystem, while brtd is a filesystem debugger, which allows file system developers to directly view the on-disk data without looking at literal ones and zeros.
In all the above projects, I’ve been incredibly surprised with the care the Rust developers have put into developing the language. I’ve used Rust as an application developer as well as a library developer. As an application developer, I’ve loved how you can write relatively well-performing code by writing idiomatic Rust; that idiomatic Rust can look and feel fairly similar to a scripting language; and the fact that if your code compiles, you’re in pretty good shape, means that systems programmers really can have nice things too.
As a library developer, I’ve loved how Rust provides powerful facilities to build abstractions that resist misuse. Ownership and borrowing are concepts that systems programmers are intimately familiar with. To have them baked into the language and enforced at compile time represents a huge leap forward in writing correct code.
How do you feel about Rust’s growth trajectory at Facebook?
I feel pretty good about Rust’s trajectory. Facebook owns a lot of physical hosts and our engineers are exposed to all sorts of problems related to systems. Rust has been fairly well received internally and is getting more attention all the time. I believe that there is and will remain enough internal expertise for other teams to confidently choose Rust for their next project.
How do you think Rust is growing as a language in 2021?
I think Rust is nearing or past the peak of its hype cycle. Personally, I think that’s a good thing. There’s that old adage: “There are only two kinds of languages: the ones people complain about and the ones nobody uses.” I don’t take complaining to necessarily be a good sign, but I do think it means people are starting to turn a more critical eye towards Rust. I believe that’s a good thing: nothing is perfect and things can only be improved if there is feedback.
Some people who have used Rust have come to really like it. Why do you think that is and what is your favorite feature about Rust?
It depends a lot on what a person is doing and in which space they’re working. For me, I love the compile time guarantees. I come from a C++ background so I’ve been used to using (and loving) ASAN (AddressSanitizer)/TSAN (ThreadSanitizer) and all the other sanitizers. What really sold Rust to me was when someone told me that Rust doesn’t (yet) have any sanitizers because it’s all enforced at compile time by the compiler.
Where can people learn more about Rust and how can they start contributing to it?
I think the best way to learn any programming language is to start writing code. Write some non-trivial code and just google search all your questions.
As for contributing: you don’t necessarily have to have any experience working on programming languages. Almost all open source projects are in great need of bug reports and testers. The easiest way to get started is to try out libraries, nightly builds, tools and report back any bugs you see.
We would like to thank Daniel, for taking the time to do this interview. It was very interesting to learn how Rust is being used as a primary language to build a wide variety of systems, tools, libraries, and services and how we are learning from all the things we build here and contributing back to the Rust community. We hope you found this interview useful and it gave you some insight into how and where Rust is being used at Facebook. Look out for more interview blogs where we meet with many more engineers and hear their thoughts on this topic.
About the Meet the Rustaceans series
Rust has consistently been ranked as the “most loved” language for the last 5 years and we at Facebook believe that Rust is an outstanding language that shines in critical issues such as memory safety, performance and reliability and is being used widely over a large range of projects here. We joined the Rust Foundation to help contribute towards the improvement and growth of Rust, which not only strengthens our commitment towards the Rust language but also towards a sustainable development of open source technologies and developer communities across the world.
This blog is a part of our Meet the Rustaceans series, where we invite the engineers and developers who use Rust on a regular basis to share their experiences and tell us about the amazing products that they are building using Rust here at Facebook. Look out for more interview blogs where we meet with many more engineers and hear their thoughts on this topic.
Interested in working in Production Engineering at Facebook? Check out our job postings on our Production Engineering career page here.
Now people can share directly to Instagram Reels from some of their favorite apps
More people are creating, sharing and watching Reels than ever before. We’ve seen the creator community dive deeply into video content – and use it to connect with their communities. We’re running a limited alpha test that lets creators share video content directly from select integrated apps to Instagram Reels. Now, creators won’t be interrupted in their workflow, making it easier for them share share and express themselves on Reels.
“With the shift to video happening across almost all online platforms, our innovative tools and services empower creativity and fuel the creator economy and we are proud to be able to offer a powerful editing tool like Videoleap that allows seamless content creation, while partnering with companies like Meta to make sharing content that much easier.”- Zeev Farbman, CEO and co-founder of Lightricks.
Starting this month, creators can share short videos directly to Instagram Reels from some of their favorite apps, including Videoleap, Reface, Smule, VivaVideo, SNOW, B612, VITA and Zoomerang, with more coming soon. These apps and others also allow direct sharing to Facebook , which is available for any business with a registered Facebook App to use.
We hope to expand this test to more partners in 2023. If you’re interested in being a part of that beta program, please fill out this form and we will keep track of your submission. We do not currently have information to share about general availability of this integration.
Learn more here about sharing Stories and Reels to Facebook and Instagram and start building today.
Q. What is the difference between the Instagram Content Publishing API and Instagram Sharing to Reels?
A: Sharing to Reels is different from the Instagram Content Publishing API, which allows Instagram Business accounts to schedule and publish posts to Instagram from third-party platforms. Sharing to Reels is specifically for mobile apps to display a ‘Share to Reels’ widget. The target audience for the Share to Reels widget is consumers, whereas the Content Publishing API is targeted towards businesses, including third-party publishing platforms such as Hootsuite and Sprout Social that consolidate sharing to social media platforms within their third-party app.
Q: Why is Instagram partnering with other apps?
A: Creators already use a variety of apps to create and edit videos before uploading them to Instagram Reels – now we’re making that experience faster and easier. We are currently doing a small test of an integration with mobile apps that creators know and love, with more coming soon.
Q: How can I share my video from another app to Reels on Instagram?
A: How it works (Make sure to update the mobile app you’re using to see the new Share to Reels option):
- Create and edit your video in one of our partner apps
- Once your video is ready, tap share and then tap the Instagram Reels icon
- You will enter the Instagram Camera, where you can customize your reel with audio, effects, Voiceover and stickers. Record any additional clips or swipe up to add an additional clip from your camera roll.
- Tap ‘Next’ to add a caption, hashtag, location, tag others or use the paid partnerships label.
- Tap ‘Share’. Your reel will be visible where you share reels today, depending on your privacy settings.
Q: How were partners selected?
A. We are currently working with a small group of developers that focus on video creation and editing as early partners. We’ll continue to expand to apps with other types of creation experiences.
Q: When will other developers be able to access Sharing to Reels on Instagram?
A: We do not currently have a date for general availability, but are planning to expand further in 2023.
Q: Can you share to Facebook Reels from other apps?
A: Yes, Facebook offers the ability for developers to integrate with Sharing to Reels. For more information on third-party sharing opportunities, check out our entire suite of sharing offerings .
First seen at developers.facebook.com
What to know about Presto SQL query engine and PrestoCon
The open source Presto SQL query engine is used by a diverse set of companies to navigate increasingly large data workflows. These companies are using Presto in support of e-commerce, cloud, security and other areas. Not only do many companies use Presto, but individuals from those companies are also active contributors to the Presto open source community.
In support of that community, Presto holds meetups around the world and has an annual conference, PrestoCon, where experts and contributors gather to exchange knowledge. This year’s PrestoCon, hosted by the Linux Foundation, takes place December 7-8 in Mountain View, CA. This blog post will explore some foundational elements of Presto and what to expect at this year’s PrestoCon.
What is Presto?
Presto is a distributed SQL query engine for data platform teams. Presto users can perform interactive queries on data where it lives using ANSI SQL across federated and diverse sources. Query engines allow data scientists and analysts to focus on building dashboards and utilizing BI tools so that data engineers can focus on storage and management, all while communicating through a unified connection layer.
In short, the scientist does not have to consider how or where data is stored, and the engineer does not have to optimize for every use case for the data sources they manage. You can learn more about Presto in a recent ELI5 video below.
Caption: Watch the video by clicking on the image above.
Presto was developed to solve the problem of petabyte-scale, multi-source data queries taking hours or days to return. These resources and time constraints make real-time analysis impossible. Presto can return results from those same queries in less than a second in most cases, allowing for interactive data exploration.
Not only is it highly scalable, but it’s also extensible, allowing you to build your own connector for any data source Presto does not already support. At a low level, Presto also supports a wide range of file types for query processing. Presto was open sourced by Meta and later donated to the Linux Foundation in September of 2019.
Here are some Presto resources for those who are new to the community:
What is PrestoCon?
PrestoCon is held annually in the Bay Area and hosted by the Linux Foundation. This year, the event takes place December 7-8 at the Computer History Museum. You can register here. Each year at PrestoCon, you can hear about the latest major evolutions of the platform, how different organizations use Presto and what plans the Technical Steering Committee has for Presto in the coming year.
Presto’s scalability is especially apparent as every year we hear from small startups, as well as industry leaders like Meta and Uber, who are using the Presto platform for different use cases, whether those are small or large. If you’re looking to contribute to open source, PrestoCon is a great opportunity for networking as well as hearing the vision that the Technical Steering Committee has for the project in the coming year.
Explore what’s happening at PrestoCon 2022:
Where is Presto used?
Since its release in November of 2013, Presto has been used as an integral part of big data pipelines within Meta and other massive-scale companies, including Uber and Twitter.
The most common use case is connecting business intelligence tools to vast data sets within an organization. This enables crucial questions to be answered faster and data-driven decision-making can be more efficient.
How does Presto work?
First, a coordinator takes your statement and parses it into a query. The internal planner generates an optimized plan as a series of stages, which are further separated into tasks. Tasks are then assigned to workers to process in parallel.
Workers then use the relevant connector to pull data from the source.
The output of each task is returned by the workers, until the stage is complete. The stage’s output is returned by the final worker towards the next stage, where another series of tasks must be executed.
The results of stages are combined, eventually returning the final result of the original statement to the coordinator, which then returns to the client.
How do I get involved?
We would love for you to join the Presto Slack channel if you have any questions or need help. Visit the community page on the Presto website to see all the ways you can get involved and find other users and developers interested in Presto.
Where can I learn more?
To learn more about Presto, check out its website for installation guides, user guides, conference talks and samples.
First seen at developers.facebook.com
How to Interpret Webhook Components in the WhatsApp Business Platform
The ways customers want to connect are changing. The WhatsApp Business Platform gives businesses an integrated way to communicate with customers right where they are. In order to integrate properly when using the Cloud API, hosted by Meta, you’ll need to leverage webhooks so applications have a way to respond to events. Webhooks allow your application to monitor three primary events on WhatsApp so you can react with different functionality depending on your goals.
This article looks at these three components, goes through the information they carry, and provides some use-case scenarios to give you an idea of the possibilities.
Interpreting Different Webhook Components
To send and receive messages on WhatsApp, it’s critical to keep track of statuses and errors to help ensure you’re communicating effectively with your customers, which you can do with webhooks.
With webhooks, the WhatsApp Business Platform monitors events and sends notifications when one occurs. These events are one of three components: messages, statuses, and errors.
Let’s explore each of these and examine examples of how you can use them.
The messages component is the largest of the three event types and contains two core objects:
Contacts — which contain information about the message’s sender.
Messages — which provide information about a message’s type and contents.
These two event types allow your application to manage and respond to people that interact with your application. The contacts object contains two pieces of information: name and WhatsApp Id. The contact’s name allows your application to use their name without further lookups. In contrast, the contact’s WhatsApp ID lets you keep track of these contacts or use the contacts/ endpoint to add additional functionality.
For instance, you can verify the customer and start the opt-in process within the customer-initiated conversation, which allows you to message them outside the initial 24-hour response window. It’s important to note that only the text, contacts, and location message types provide contact information.
The message object is where the bulk of the information is stored, including the message contents, type of message, and other relevant information. Depending on the message type, the actual payload of the message component can vary widely. It’s crucial to determine the message type to understand the potential payload. Message types include:
Text: a standard text-only message
Contact: contains a user’s full contact details
Location: address, latitude, and longitude
Unknown: unsupported messages from users, which usually contain errors.
Ephemeral: disappearing messages
Media message types: contain information for the specified media file. These types include:
These different data types can have very different uses, from reviewing images and screenshots from concerned customers to collecting information about where to ship goods and send services. To use these different data types most effectively, you can create applications to handle different forms of communication, with functionalities such as:
Ask your customers to provide a shipping or mailing address. You can use the location-based message feature to capture your users’ location to determine where to send their goods and services.
Show customers products and communicate product details through a message. You can use the referred_product field within messages to offer your users specific product details. Using this field develops a more personal, conversational shopping experience and customer interactions.
Build support functionality that allows customers to take and send images and videos of product concerns, and submit those for a support case. Once the user has submitted a support ticket, the app can track the case — including steps taken towards resolution and conversations between support teams and the customer through WhatsApp — using a unique case identifier.
These are just some potential features you can build using the interactivity provided by webhooks and the message object. These features extend your current communication channels and provide additional options for customers.
Where the messages component provides your application with insight into events that originate directly from your customers, the statuses component keeps track of the results of messages you send and the conversation history. There are six status components:
- Sent: the application sent your message and is in transit.
- Delivered: the user’s device successfully received the message.
- Read: the user has read your message.
- Deleted: a user deleted a message that you sent.
- Warning: a message sent by your application contains an item that isn’t available or doesn’t exist.
- Failed: a message sent by your application failed to arrive.
Status components also contain information on the recipient ID, the conversation, and the pricing related to the current conversation. Conversations on WhatsApp are a grouping of messages within a 24-hour window that are either user-initiated or business-initiated. Keeping track of these conversations is vital, as a new conversation occurs when you send additional responses after the 24-hour period ends.
Some functionality you may want to add to your application based on status events includes:
- Ensuring your application has sent generated messages, they arrived, and the recipient potentially read them by using a combination of these status types and timestamps within the status object. This information allows your application to follow up with customers if they didn’t engage.
- Keep analytical information about your application’s messages, especially regarding business-initiated conversations. For example, if your application uses a WhatsApp customer contact list to send offer messages, the status component helps you understand how many were sent, delivered, read, responded to, or failed to measure your campaign’s success.
Finally, the errors component allows your application to receive any out-of-band errors within WhatsApp that affect your platform. These errors don’t stop your application from compiling or working but are typically caused when your application is misusing specific functionality. The following are some typical errors.
Error Code 368, Temporarily Blocked for Policy Violations
If your application violates WhatsApp Business Messaging or Commerce policy, your account may be temporarily banned. You can monitor this and pause your application while troubleshooting.
Error 506, Duplicate Post
If your workflows unintentionally generate duplicate messages, you can monitor this to find the source.
Error 131043, Message Expired
Sometimes, messages are not sent during their time to live (TTL) duration. Use this code to know which messages to schedule for resending if needed.
Error handling is a broad, complex subject, and there are many other use cases for which you should be implementing error handling. The errors component helps extend your error handling on the WhatsApp Business Platform for greater consistency.
This article took a high-level look at messages, statuses, and errors returned by webhooks and explored ways you can use these three components to expand your application’s functionality.
Messages provide information on customer interactions, statuses give insight into messages your app sends, and error notices enable you to increase your application’s resilience. Webhooks are critical to ensuring your app interacts with customers seamlessly.
The WhatsApp Business Platform’s webhooks provide your applications with real-time data, enabling you to build better experiences as you interact with customers. Ready to know more? Dive deeper into everything the WhatsApp Business Platform has to offer.
First seen at developers.facebook.com
Twitter Users May Witness Dip in Followers as Elon Musk Has New Plans for Spams
WhatsApp Bans Over 23 Lakh Accounts in India in October, ‘Proactively’ Barred 8.1 Lakh Users
Mark Zuckerberg Calls Apple’s App Store Moderation Rules a ‘Conflict of Interest’
Twitter Faces Ban Over Content Moderation, EU Chief Warns Elon Musk: Report
How to Create Effective Social Media Guidelines for Your Business
Building LinkedIn’s Skills Graph to Power a Skills-First World
(Re)building Threat Detection and Incident Response at LinkedIn
Access Levels for Gaming Apps
Plug and Play LA announces program with Meta to Transform the Future of Business Messaging
Introducing the ‘Instagram Explore home’ Ads Placements via the Instagram Marketing API
Twitter’s Chief of Information Security Lea Kissner Exists Company Amid Mass Layoffs
Get Started with the Page Insights API
FACEBOOK1 week ago
How to Send Interactive Messages with the WhatsApp Business Platform
FACEBOOK1 week ago
Hermit: Deterministic Linux for Controlled Testing and Software Bug-finding
OTHER1 week ago
Elon Musk Urged by US Senator to Better Protect US Users’ Data After Whistleblower Testimony
FACEBOOK1 week ago
Building Intuitive Interactions in VR: Interaction SDK, First Hand Showcase and Other Resources
OTHER2 weeks ago
Elon Musk Says Twitter Managers Can Approve Remote Work for Staff at Their Own Risk
OTHER7 days ago
Twitter, Other Social Media Apps Fail to Remove Hate Speech, Says EU Review
FACEBOOK3 days ago
How to Interpret Webhook Components in the WhatsApp Business Platform
OTHER2 weeks ago
Twitter Sued by Disabled Employee Over Elon Musk’s Ban on Remote Work