We recommend Captum to model developers looking to improve their work, interpretability researchers focused on multi-modal models, and application engineers who are using trained models in production.
Where is Captum used?
Captum was open sourced in mid-October 2019 during the PyTorch Developer Conference.
Within Facebook, we use Captum to better understand how our AI models interpret Profile and other Facebook pages, which often host a variety of text, audio, picture, video, and linked content.
Where can I learn more?
If you have any further questions about Captum, let us know on our YouTube channel, or by tweeting at us. We always want to hear from you and hope you will find this open source project and the new ELI5 series useful.
About the ELI5 series
In a series of short videos (~1 min in length), one of our Developer Advocates on the Facebook Open Source team explains a Facebook open source project in a way that is easy to understand and use.
We will write an accompanying blog post (like the one you’re reading right now) for each of these videos, which you can find on our YouTube channel.
Interested in working with open source at Facebook? Check out our open source-related job postings on our career page by taking this quick survey.
Meet the Developers – Linux Kernel Team (David Vernet)
Credit: Larry Ewing (firstname.lastname@example.org) and The GIMP for the original design of Tux the penguin.
For today’s interview, we have David Vernet, a core systems engineer on the Kernel team at Meta. He works on the BPF (Berkeley Packet Filter) and the Linux kernel scheduler. This series highlights Meta Software Engineers who contribute to the Linux kernel. The Meta Linux Kernel team works with the broader Linux community to add new features to the kernel and makes sure that the kernel works well in Meta production data centers. Engineers on the team work with peers in the industry to make the kernel better for Meta’s workloads and to make Linux better for everyone.
Tell us about yourself.
I’m a systems engineer who’s spent a good chunk of his career in the kernel space, and some time in the user-space as well working on a microkernel. Right now, I’m focusing most of my time on BPF and the Linux kernel scheduler.
I started my career as a web developer after getting a degree in math. After going to grad school, I realized that I was happiest when hacking on low-level systems and figuring out how computers work.
As a kernel developer at Meta, what does your typical day look like?
I’m not a maintainer of any subsystems in the kernel, so my typical day is filled with almost exclusively coding and engineering. That being said, participating in the upstream Linux kernel community is one of the coolest parts of being on the kernel team, so I still spend some time reading over upstream discussions. A typical day goes something like this:
Hack on the project that I’m working on. Lately, that’s adding a user-space ringbuffer map type to BPF.
Work on drafting an article for lwn.net.
What have you been excited about or incredibly proud of lately?
I recently submitted a patch-set to enable a new map type in BPF. This allows user-space to publish messages to BPF programs in the kernel over the ringbuffer. This map type is exciting because it sets the stage to enable frameworks for user-space to drive logic in BPF programs in a performant way.
Is there something especially exciting about being a kernel developer at a company like Meta?
The Meta kernel team has a strong upstream-first culture. Bug fixes that we find in our Meta kernel, and features that we’d like to add, are almost always first submitted to the upstream kernel, and then they are backported to our internal kernel.
Do you have a favorite part of the kernel dev life cycle?
I enjoy architecting and designing APIs. Kernel code can never crash and needs to be able to run forever. I find it gratifying to architect systems in the kernel that make it easy to reason about correctness and robustness and provide intuitive APIs that make it easy for other parts of the kernel to use your code.
I also enjoy iterating with the upstream community. It’s great that your patches have a whole community of people looking at them to help you find bugs in your code and suggest improvements that you may never have considered on your own. A lot of people find this process to be cumbersome, but I find that it’s a small price to pay for what you get out of it.
Tell us a bit about the topic you presented at the Linux Plumbers Conference this year.
We presented the live patch feature in the Linux kernel, describing how we have utilized it at Meta and how our hyper-scale has shown some unique challenges with the feature.
What are some of the misconceptions about kernel or open source software development that you have encountered in your career?
The biggest misconception is that it’s an exclusive, invite-only club to contribute to the Linux kernel. You certainly must understand operating systems to be an effective contributor and be ready to receive constructive criticism when there is scope for improvement in your code. Still, the community always welcomes people who come in with an open mind and want to contribute.
What resources are helpful in getting started in kernel development?
There is a lot of information out there that people have written on how to get integrated into the Linux kernel community. I wrote a blog post on how to get plugged into Linux kernel upstream mailing list discussions, and another on how to submit your first patch. There is also a video on writing and submitting your first Linux kernel patch from Greg Kroah-Hartman.
In terms of resources to learn about the kernel itself, there are many resources and books, such as:
- Linux Weekly News
- Linux Kernel Programming (parts 1 and 2)
- Linux Kernel in a Nutshell: A Desktop Quick Reference
- Our public blog posts
Where can people find you and follow your work?
I have a blog where I talk about my experiences as a systems engineer: https://www.bytelab.codes/. I publish articles that range from topics that are totally newcomer friendly to more advanced topics that discuss kernel code in more detail. Feel free to check it out and let me know if there’s anything you’d like me to discuss.
First seen at developers.facebook.com
Get started with WhatsApp Business Platform in Minutes with Postman
Our collaboration brings tools you already use to WhatsApp Business Platforms APIs
Postman is a best-in-class API platform used by 20M developers worldwide. Using Postman simplifies each step of the API lifecycle and streamlines collaboration.
Postman’s strong platform and broad adoption in the developer community made deciding to work with Postman to deliver a robust developer experience an easy decision for our WhatsApp Business Platform product team.
What Postman means for your WhatsApp projects
The benefits of this collaboration for developers are clear – you can easily leverage Postman’s platform with your Meta projects to onboard, collaborate, and contribute towards documentation and best practices as you build out your integrations.
The WhatsApp team is able to offer, via Postman, an API collection that pre-fills environment variables and walks you through your initial test requests – helping developers dive right in to using the Cloud API. Our product managers show you how easy it is to get started with Postman in this session from Conversations:
The public Postman workspace fosters collaboration – allowing environments, collections, and documentation augmentation to happen in one place.
Postman’s API documentation tools augment our own documentation and allows developers to contribute directly to the community’s shared knowledge, building a strong reference library for all developers and encouraging new, innovative use cases.
Working with Postman from the beginning helps create a developer-friendly experience for the WhatsApp Business Platform – allowing you to get started quickly, build community, and share knowledge.
Want to know more about our partnership with Postman? Check out their case study, follow along with the video above, or dive right into the Postman Workspace for the WhatsApp Business Platform.
First seen at developers.facebook.com
Summer of open source: building more efficient AI with PyTorch
Note: Special thanks to Less Wright, Partner Engineer, Meta AI, for review of and additional insights into the post.
This post on creating efficient artificial intelligence (AI) is the second in the “Summer of open source” series. This series aims to provide a handful of useful resources and learning content in areas where open source projects are creating impact across Meta and beyond. Follow along as we explore other areas where Meta Open Source is moving the industry forward by sharing innovative, scalable tools.
PyTorch: from foundational technology to foundation
Since its initial release in 2016, PyTorch has been widely used in the deep learning community, and its roots in research are now consistently expanding for use in production scenarios. In an exciting time for machine learning (ML) and artificial intelligence (AI), where novel methods and use cases for AI models continue to expand, PyTorch has reached the next chapter in its history as it moves to the newly established, independent PyTorch Foundation under the Linux Foundation umbrella. The foundation is made up of a diverse governing board including representatives from AMD, Amazon Web Services, Google Cloud, Microsoft Azure and Nvidia, and the board is intended to expand over time. The mission includes driving adoption of AI tooling through vendor-neutral projects and making open source tools, libraries and other components accessible to everyone. The move to the foundation will also enable PyTorch and its open source community to continue to accelerate the path from prototyping to production for AI and ML.
Streamlining AI processes with Meta open source
PyTorch is a great example of the power of open source. As one of the early open source deep learning frameworks, PyTorch has allowed people from across disciplines to experiment with deep learning and apply their work in wide-ranging fields. PyTorch supports everything from experiments in search applications to autonomous vehicle development to ground-penetrating radar, and these are only a few of its more recent applications. Pairing a versatile library of AI tools with the open source community unlocks the ability to quickly iterate on and adapt technology at scale for many different uses.
As AI is being implemented more broadly, models are trending up in size to tackle more complex problems, but this also means that the resources needed to train these models have increased substantially. Fortunately, many folks in the developer community have recognized the need for models to use fewer resources—both from a practical and environmental standpoint. This post will explore why quantization and other types of model compression can be a catalyst for efficient AI.
Establishing a baseline for using PyTorch
Most of this post explores some intermediate and advanced features of PyTorch. If you are a beginner that is looking to get started, or an expert that is currently using another library, it’s easiest to get started with some basics. Check out the beginner’s guide to PyTorch, which includes an introduction to a complete ML workflow using the Fashion MNIST dataset.
Here are some other resources that you might check out if you’re new to PyTorch:
- PyTorch Community Stories: Learn how PyTorch is making an impact across different industries like agriculture, education, travel and others
- PyTorch Beginner Series: Explore a video playlist of fundamental techniques including getting started with tensors, building models, training and inference in PyTorch.
Quantization: Applying time-tested techniques to AI
There are many pathways to making AI more efficient. Codesigning hardware and software to optimize for AI can be highly effective, but bespoke hardware-software solutions take considerable time and resources to develop. Creating faster and smaller architectures is another path to efficiency, but many of these architectures suffer from accuracy loss when compared to larger models, at least for the time being. A simpler approach is to find ways of reducing the resources that are needed to train and serve existing models. In PyTorch, one way to do that is through model compression using quantization.
Quantization is a mathematical technique that has been used to create lossy digital music files and convert analog signals to digital ones. By executing mathematical calculations with reduced precision, quantization allows for significantly higher performance on many hardware platforms. So why use quantization to make AI more efficient? Results show that in certain cases, using this relatively simple technique can result in dramatic speedups (2-4 times) for model inference.
The parameters that make up a deep learning model are typically decimal numbers in floating point (FP) precision; each parameter requires either 16 bits or 32 bits of memory. When using quantization, numbers are often converted to INT4 or INT8, which occupy only 4 or 8 bits. This reduces how much memory models require. Additionally, chip manufacturers include special arithmetic that makes operations using integers faster than using decimals.
There are 3 methods of quantization that can be used for training models: dynamic, static and quantize-aware training (QAT). A brief overview of the benefits and weaknesses is described in the table below. To learn how to implement each of these in your AI workflows, read the Practical Quantization in PyTorch blog post.
Additional overhead in every forward pass
May need regular recalibration for distribution drift
Quantize-Aware Training (QAT)
High computational cost
Additional features for speeding up your AI workflow
Quantization isn’t the only way to make PyTorch-powered AI more efficient. Features are updated regularly, and below are a few other ways that PyTorch can improve AI workflows:
Inference mode: This mode can be used for writing PyTorch code if you’re only using the code for running inference. Inference mode changes some of the assumptions when working with tensors to speed up inference. By telling PyTorch that you won’t use tensors for certain applications later (in this case, autograd), it adjusts to make code run faster in these specific scenarios.
Low precision: Quantization works only at inference time, that is, after you have trained your model. For the training process itself, PyTorch uses AMP, or automatic mixed precision training, to find the best format based on which tensors are used (FP16, FP32 or BF16). Low-precision deep learning in PyTorch has several advantages. It can help lower the size of a model, reduce the memory that is required to train models and decrease the power that is needed to run models. To learn more, check out this tutorial for using AMP with CUDA-capable GPUs.
Channels last: When it comes to vision models, NHWC, otherwise known as channels-last, is a faster tensor memory format in PyTorch. Having data stored in the channels-last format accelerates operations in PyTorch. Formatting input tensors as channels-last reduces the overhead that is needed for conversion between different format types, resulting in faster inference.
Optimize for inference: This TorchScript prototype implements some generic optimizations that should speed up models in all environments, and it can also prepare models for inference with build-specific settings. Primary use cases include vision models on CPUs (and GPUs) at this point. Since this is a prototype, it’s possible that you may run into issues. Raise an issue that occurs on the PyTorch GitHub repository.
Unlocking new potential in PyTorch
Novel methods for accelerating AI workflows are regularly explored on the PyTorch blog. It’s a great place to keep up with techniques like the recent BetterTransformer, which increases speedup and throughput in Transformer models by up to 2 times for common execution scenarios. If you’re interested in learning how to implement specific features in PyTorch, the recipes page allows you to search by categories like model optimization, distributed training and interpretability. This post is only a sampling of how tools like PyTorch are moving open source and AI forward.
To stay up to date with the latest in Meta Open Source for artificial intelligence and machine learning, visit our open source site, subscribe to our YouTube channel, or follow us on Facebook, Twitter and LinkedIn.
First seen at developers.facebook.com
How Brands Can Support Indigenous Communities on Social—the Right Way
US Senate Panel Approves Bill Empowering News Organisations to Negotiate With Facebook, Google for Revenue
WhatsApp Working to Keep Iranians Connected Amid Widespread Internet Shutdown Over Nationwide Protests
Facebook Whistleblower Frances Haugen Launches ‘Beyond the Screen’ Organisation to Tackle Social Media Harms
How To Win at TikTok (According to TikTok)
Meta for Business: How To Get the Best Results From Each Platform
Twitter Expanding Birdwatch Community Fact-Checking Programme With New Onboarding Process, More
Twitter Ordered to Give Information to Elon Musk Regarding Spam and Bot Accounts
Operating system upgrades at LinkedIn’s scale
Career stories: Rejoining LinkedIn to scale our media infrastructure
14 Best Apps For Instagram Collages in 2022
How to Write Your Best Social Media Bio [+28 Free Templates]
LINKEDIN2 weeks ago
Real-time analytics on network flow data with Apache Pinot
FACEBOOK1 week ago
Introducing Facebook Graph API v15.0 and Marketing API v15.0
FACEBOOK2 weeks ago
Meet the Developers: Linux Kernel Team (Jonathan Zhang)
Uncategorized1 week ago
The 12 Best Chatbot Examples for Businesses
LINKEDIN2 weeks ago
Feathr joins LF AI & Data Foundation
Uncategorized1 week ago
I Tried Instagram Automation (So You Don’t Have To): An Experiment
Uncategorized1 week ago
Experiment: What Reels Caption Length Gets the Best Engagement?
OTHER1 week ago
Zoom Resolves Connectivity Issues After Over 40,000 Users Reported Problem