This article was written in collaboration with Gus Wynn, a Production Engineer at Facebook.
For today’s interview, we have Gus Wynn who is a Production Engineer on the hardware health diagnosis team at Facebook. The hardware health diagnosis team works on providing a platform for the collection and analysis of hardware-related data at scale. While working on this team, Gus has been using Rust as one of the main languages for development. Most of his work these days centers around building large scale distributed systems to perform hardware health diagnosis. Let’s hear from him about how his experience with Rust has been and learn more about his work.
In what capacity have you been using Rust?
My team is building a large scale distributed system in Rust to perform hardware health diagnosis. I write Rust code as my primary language on most days. While I spend a lot of time working on those distributed systems, I end up using my time to work on the libraries, bindings and common code required to build an at-scale system like this at Facebook. In particular, I have been interested in the performance of async rust, and have been trying to make it easier for others to write fast code easily.
Why did your team at Facebook choose to use Rust over other languages?
We chose Rust because it combines a large number of desirable properties:
- The strong type system in Rust lets us model complex problems and ensures the correctness of our implementations statically with the compiler.
- Rust is fast, by default, with very little effort.
- Memory safety is a guarantee in Rust. This, when coupled with it’s speed, is a property unique to Rust, and it saves us an absurd amount of time from debugging problems that simply do not exist in Rust as compared to other languages.
What are some of the projects that you’ve worked on at Facebook that use Rust?
Some of the projects I have worked on that use Rust include:
- Core RPC (Remote Procedure Call) libraries.
- Pieces of the hardware diagnosis system that my team is building. This system will enable us to remotely diagnose issues in the machines that are broken.
- Core datastore client libraries.
Apart from this, one of my favorite things to work on in Rust is actually teaching it. I love answering questions, reviewing code, unblocking people who might be confused and creating best practices.
How do you feel about Rust’s growth trajectory at Facebook?
I think the trajectory of Rust’s growth at Facebook is extremely positive. I regularly see people try it out and immediately understand the benefit. Rust is an incredible tool that makes writing correct, reliable and fast software a lot easier. In fact, it makes it so much easier, that people very quickly feel that it’s worth the activation energy required to learn a new language and build the required shared libraries.
What value does Facebook add to Rust?
Facebook has produced and helped maintain some very nice libraries. Off the top of my head, there is Gazebo, and cargo-guppy, but the one that is a force-multiplier for me basically everyday, as well as a force-multiplier for many people outside of FB, is CXX, which makes it incredibly easy to bind C++ and Rust code together. I believe this is a fundamental piece to continuing Rust’s growth trajectory in and outside of Facebook.
How do you think Rust is growing as a language in 2021?
I have seen Rust continue to grow at a faster and faster rate for years. I think that the work that the open source teams are doing, both on the technical side as well as the community work, is incredible. In particular, I think the Async Visions project is an incredible idea, and I can’t wait for it to be even easier for Rust to replace many, many high-performance server uses!
Some people who have used Rust have come to really like it. Why do you think that is and what is your favorite feature about Rust?
This is very hard to answer because Rust is an incredibly well designed language. I think my favorite part is its type system, because:
- Enums-with-data, also called sum types, or tagged unions, make it easy to model real-world problems. The best of these are Result and Option!
- Traits are a better way than inheritance to express relationships.
- It’s a strong enough type system for type inference to make it easy to read and write code.
- The type system is secretly the way that Rust implements its memory safety and multi-threaded race prevention. For example, the lifetimes are part of the type system, and the auto-traits Send and Sync are used to encode the necessary info to prevent race-conditions.
Where can people learn more about Rust and how can they start contributing to it?
I would definitely suggest starting with The Book-Rust Programming Language. There are also other great resources for more intermediate or advanced content. In particular, Jon Gjengset’s live streams and his early-access book are incredible for getting to the next level!
We would like to thank Gus for taking the time to do this interview. It was very interesting to learn how Rust is being used as a primary language for analysis of hardware-related data at scale and how we are learning from all the things we build here and contributing back to the Rust community. We hope you found this interview useful and it gave you some insight into how and where Rust is being used at Facebook.
Check out our previous blogs from the Meet the Rustaceans series:
Meet the Rustaceans: Chris Konstad
Meet the Rustaceans: Pedro Rittner
Meet the Rustaceans: Neil Mitchell
Meet the Rustaceans: Daniel Xu
Meet the Rustaceans: Eric Kuecks
About the Meet the Rustaceans series
Rust has consistently been ranked as the “most loved” language for the last 5 years and we at Facebook believe that Rust is an outstanding language that shines in critical issues such as memory safety, performance and reliability and is being used widely over a large range of projects here. We joined the Rust Foundation to help contribute towards the improvement and growth of Rust, which not only strengthens our commitment towards the Rust language but also towards a sustainable development of open source technologies and developer communities across the world.
This blog is a part of our Meet the Rustaceans series, where we invite the engineers and developers who use Rust on a regular basis to share their experiences and tell us about the amazing products that they are building using Rust here at Facebook. Look out for more interview blogs where we meet with many more engineers and hear their thoughts on this topic.
Interested in working in Production Engineering at Facebook? Check out our job postings on our Production Engineering career page here.
Meet the Developers – Linux Kernel Team (David Vernet)
Credit: Larry Ewing (email@example.com) and The GIMP for the original design of Tux the penguin.
For today’s interview, we have David Vernet, a core systems engineer on the Kernel team at Meta. He works on the BPF (Berkeley Packet Filter) and the Linux kernel scheduler. This series highlights Meta Software Engineers who contribute to the Linux kernel. The Meta Linux Kernel team works with the broader Linux community to add new features to the kernel and makes sure that the kernel works well in Meta production data centers. Engineers on the team work with peers in the industry to make the kernel better for Meta’s workloads and to make Linux better for everyone.
Tell us about yourself.
I’m a systems engineer who’s spent a good chunk of his career in the kernel space, and some time in the user-space as well working on a microkernel. Right now, I’m focusing most of my time on BPF and the Linux kernel scheduler.
I started my career as a web developer after getting a degree in math. After going to grad school, I realized that I was happiest when hacking on low-level systems and figuring out how computers work.
As a kernel developer at Meta, what does your typical day look like?
I’m not a maintainer of any subsystems in the kernel, so my typical day is filled with almost exclusively coding and engineering. That being said, participating in the upstream Linux kernel community is one of the coolest parts of being on the kernel team, so I still spend some time reading over upstream discussions. A typical day goes something like this:
Hack on the project that I’m working on. Lately, that’s adding a user-space ringbuffer map type to BPF.
Work on drafting an article for lwn.net.
What have you been excited about or incredibly proud of lately?
I recently submitted a patch-set to enable a new map type in BPF. This allows user-space to publish messages to BPF programs in the kernel over the ringbuffer. This map type is exciting because it sets the stage to enable frameworks for user-space to drive logic in BPF programs in a performant way.
Is there something especially exciting about being a kernel developer at a company like Meta?
The Meta kernel team has a strong upstream-first culture. Bug fixes that we find in our Meta kernel, and features that we’d like to add, are almost always first submitted to the upstream kernel, and then they are backported to our internal kernel.
Do you have a favorite part of the kernel dev life cycle?
I enjoy architecting and designing APIs. Kernel code can never crash and needs to be able to run forever. I find it gratifying to architect systems in the kernel that make it easy to reason about correctness and robustness and provide intuitive APIs that make it easy for other parts of the kernel to use your code.
I also enjoy iterating with the upstream community. It’s great that your patches have a whole community of people looking at them to help you find bugs in your code and suggest improvements that you may never have considered on your own. A lot of people find this process to be cumbersome, but I find that it’s a small price to pay for what you get out of it.
Tell us a bit about the topic you presented at the Linux Plumbers Conference this year.
We presented the live patch feature in the Linux kernel, describing how we have utilized it at Meta and how our hyper-scale has shown some unique challenges with the feature.
What are some of the misconceptions about kernel or open source software development that you have encountered in your career?
The biggest misconception is that it’s an exclusive, invite-only club to contribute to the Linux kernel. You certainly must understand operating systems to be an effective contributor and be ready to receive constructive criticism when there is scope for improvement in your code. Still, the community always welcomes people who come in with an open mind and want to contribute.
What resources are helpful in getting started in kernel development?
There is a lot of information out there that people have written on how to get integrated into the Linux kernel community. I wrote a blog post on how to get plugged into Linux kernel upstream mailing list discussions, and another on how to submit your first patch. There is also a video on writing and submitting your first Linux kernel patch from Greg Kroah-Hartman.
In terms of resources to learn about the kernel itself, there are many resources and books, such as:
- Linux Weekly News
- Linux Kernel Programming (parts 1 and 2)
- Linux Kernel in a Nutshell: A Desktop Quick Reference
- Our public blog posts
Where can people find you and follow your work?
I have a blog where I talk about my experiences as a systems engineer: https://www.bytelab.codes/. I publish articles that range from topics that are totally newcomer friendly to more advanced topics that discuss kernel code in more detail. Feel free to check it out and let me know if there’s anything you’d like me to discuss.
First seen at developers.facebook.com
Get started with WhatsApp Business Platform in Minutes with Postman
Our collaboration brings tools you already use to WhatsApp Business Platforms APIs
Postman is a best-in-class API platform used by 20M developers worldwide. Using Postman simplifies each step of the API lifecycle and streamlines collaboration.
Postman’s strong platform and broad adoption in the developer community made deciding to work with Postman to deliver a robust developer experience an easy decision for our WhatsApp Business Platform product team.
What Postman means for your WhatsApp projects
The benefits of this collaboration for developers are clear – you can easily leverage Postman’s platform with your Meta projects to onboard, collaborate, and contribute towards documentation and best practices as you build out your integrations.
The WhatsApp team is able to offer, via Postman, an API collection that pre-fills environment variables and walks you through your initial test requests – helping developers dive right in to using the Cloud API. Our product managers show you how easy it is to get started with Postman in this session from Conversations:
The public Postman workspace fosters collaboration – allowing environments, collections, and documentation augmentation to happen in one place.
Postman’s API documentation tools augment our own documentation and allows developers to contribute directly to the community’s shared knowledge, building a strong reference library for all developers and encouraging new, innovative use cases.
Working with Postman from the beginning helps create a developer-friendly experience for the WhatsApp Business Platform – allowing you to get started quickly, build community, and share knowledge.
Want to know more about our partnership with Postman? Check out their case study, follow along with the video above, or dive right into the Postman Workspace for the WhatsApp Business Platform.
First seen at developers.facebook.com
Summer of open source: building more efficient AI with PyTorch
Note: Special thanks to Less Wright, Partner Engineer, Meta AI, for review of and additional insights into the post.
This post on creating efficient artificial intelligence (AI) is the second in the “Summer of open source” series. This series aims to provide a handful of useful resources and learning content in areas where open source projects are creating impact across Meta and beyond. Follow along as we explore other areas where Meta Open Source is moving the industry forward by sharing innovative, scalable tools.
PyTorch: from foundational technology to foundation
Since its initial release in 2016, PyTorch has been widely used in the deep learning community, and its roots in research are now consistently expanding for use in production scenarios. In an exciting time for machine learning (ML) and artificial intelligence (AI), where novel methods and use cases for AI models continue to expand, PyTorch has reached the next chapter in its history as it moves to the newly established, independent PyTorch Foundation under the Linux Foundation umbrella. The foundation is made up of a diverse governing board including representatives from AMD, Amazon Web Services, Google Cloud, Microsoft Azure and Nvidia, and the board is intended to expand over time. The mission includes driving adoption of AI tooling through vendor-neutral projects and making open source tools, libraries and other components accessible to everyone. The move to the foundation will also enable PyTorch and its open source community to continue to accelerate the path from prototyping to production for AI and ML.
Streamlining AI processes with Meta open source
PyTorch is a great example of the power of open source. As one of the early open source deep learning frameworks, PyTorch has allowed people from across disciplines to experiment with deep learning and apply their work in wide-ranging fields. PyTorch supports everything from experiments in search applications to autonomous vehicle development to ground-penetrating radar, and these are only a few of its more recent applications. Pairing a versatile library of AI tools with the open source community unlocks the ability to quickly iterate on and adapt technology at scale for many different uses.
As AI is being implemented more broadly, models are trending up in size to tackle more complex problems, but this also means that the resources needed to train these models have increased substantially. Fortunately, many folks in the developer community have recognized the need for models to use fewer resources—both from a practical and environmental standpoint. This post will explore why quantization and other types of model compression can be a catalyst for efficient AI.
Establishing a baseline for using PyTorch
Most of this post explores some intermediate and advanced features of PyTorch. If you are a beginner that is looking to get started, or an expert that is currently using another library, it’s easiest to get started with some basics. Check out the beginner’s guide to PyTorch, which includes an introduction to a complete ML workflow using the Fashion MNIST dataset.
Here are some other resources that you might check out if you’re new to PyTorch:
- PyTorch Community Stories: Learn how PyTorch is making an impact across different industries like agriculture, education, travel and others
- PyTorch Beginner Series: Explore a video playlist of fundamental techniques including getting started with tensors, building models, training and inference in PyTorch.
Quantization: Applying time-tested techniques to AI
There are many pathways to making AI more efficient. Codesigning hardware and software to optimize for AI can be highly effective, but bespoke hardware-software solutions take considerable time and resources to develop. Creating faster and smaller architectures is another path to efficiency, but many of these architectures suffer from accuracy loss when compared to larger models, at least for the time being. A simpler approach is to find ways of reducing the resources that are needed to train and serve existing models. In PyTorch, one way to do that is through model compression using quantization.
Quantization is a mathematical technique that has been used to create lossy digital music files and convert analog signals to digital ones. By executing mathematical calculations with reduced precision, quantization allows for significantly higher performance on many hardware platforms. So why use quantization to make AI more efficient? Results show that in certain cases, using this relatively simple technique can result in dramatic speedups (2-4 times) for model inference.
The parameters that make up a deep learning model are typically decimal numbers in floating point (FP) precision; each parameter requires either 16 bits or 32 bits of memory. When using quantization, numbers are often converted to INT4 or INT8, which occupy only 4 or 8 bits. This reduces how much memory models require. Additionally, chip manufacturers include special arithmetic that makes operations using integers faster than using decimals.
There are 3 methods of quantization that can be used for training models: dynamic, static and quantize-aware training (QAT). A brief overview of the benefits and weaknesses is described in the table below. To learn how to implement each of these in your AI workflows, read the Practical Quantization in PyTorch blog post.
Additional overhead in every forward pass
May need regular recalibration for distribution drift
Quantize-Aware Training (QAT)
High computational cost
Additional features for speeding up your AI workflow
Quantization isn’t the only way to make PyTorch-powered AI more efficient. Features are updated regularly, and below are a few other ways that PyTorch can improve AI workflows:
Inference mode: This mode can be used for writing PyTorch code if you’re only using the code for running inference. Inference mode changes some of the assumptions when working with tensors to speed up inference. By telling PyTorch that you won’t use tensors for certain applications later (in this case, autograd), it adjusts to make code run faster in these specific scenarios.
Low precision: Quantization works only at inference time, that is, after you have trained your model. For the training process itself, PyTorch uses AMP, or automatic mixed precision training, to find the best format based on which tensors are used (FP16, FP32 or BF16). Low-precision deep learning in PyTorch has several advantages. It can help lower the size of a model, reduce the memory that is required to train models and decrease the power that is needed to run models. To learn more, check out this tutorial for using AMP with CUDA-capable GPUs.
Channels last: When it comes to vision models, NHWC, otherwise known as channels-last, is a faster tensor memory format in PyTorch. Having data stored in the channels-last format accelerates operations in PyTorch. Formatting input tensors as channels-last reduces the overhead that is needed for conversion between different format types, resulting in faster inference.
Optimize for inference: This TorchScript prototype implements some generic optimizations that should speed up models in all environments, and it can also prepare models for inference with build-specific settings. Primary use cases include vision models on CPUs (and GPUs) at this point. Since this is a prototype, it’s possible that you may run into issues. Raise an issue that occurs on the PyTorch GitHub repository.
Unlocking new potential in PyTorch
Novel methods for accelerating AI workflows are regularly explored on the PyTorch blog. It’s a great place to keep up with techniques like the recent BetterTransformer, which increases speedup and throughput in Transformer models by up to 2 times for common execution scenarios. If you’re interested in learning how to implement specific features in PyTorch, the recipes page allows you to search by categories like model optimization, distributed training and interpretability. This post is only a sampling of how tools like PyTorch are moving open source and AI forward.
To stay up to date with the latest in Meta Open Source for artificial intelligence and machine learning, visit our open source site, subscribe to our YouTube channel, or follow us on Facebook, Twitter and LinkedIn.
First seen at developers.facebook.com
How Brands Can Support Indigenous Communities on Social—the Right Way
US Senate Panel Approves Bill Empowering News Organisations to Negotiate With Facebook, Google for Revenue
WhatsApp Working to Keep Iranians Connected Amid Widespread Internet Shutdown Over Nationwide Protests
Facebook Whistleblower Frances Haugen Launches ‘Beyond the Screen’ Organisation to Tackle Social Media Harms
How To Win at TikTok (According to TikTok)
Meta for Business: How To Get the Best Results From Each Platform
Twitter Expanding Birdwatch Community Fact-Checking Programme With New Onboarding Process, More
Twitter Ordered to Give Information to Elon Musk Regarding Spam and Bot Accounts
Operating system upgrades at LinkedIn’s scale
Career stories: Rejoining LinkedIn to scale our media infrastructure
14 Best Apps For Instagram Collages in 2022
How to Write Your Best Social Media Bio [+28 Free Templates]
LINKEDIN2 weeks ago
Real-time analytics on network flow data with Apache Pinot
FACEBOOK1 week ago
Introducing Facebook Graph API v15.0 and Marketing API v15.0
FACEBOOK2 weeks ago
Meet the Developers: Linux Kernel Team (Jonathan Zhang)
Uncategorized1 week ago
The 12 Best Chatbot Examples for Businesses
LINKEDIN2 weeks ago
Feathr joins LF AI & Data Foundation
Uncategorized1 week ago
I Tried Instagram Automation (So You Don’t Have To): An Experiment
FACEBOOK4 days ago
Summer of open source: building more efficient AI with PyTorch
OTHER1 week ago
Zoom Resolves Connectivity Issues After Over 40,000 Users Reported Problem