Posted by means of Caren Chang – Developer Members of the family Engineer

The Jetpack Media3 library allows Android apps to construct prime quality media apps. As a part of the Media3 library, the Transformer module targets to supply simple to make use of, dependable, and performant APIs for transcoding and enhancing media.

For instance, apps can use Transformer to use enhancing operations equivalent to trimming a protracted piece of media document, or making use of results to video tracks. Transformer will also be used to transform media recordsdata from one structure to every other, equivalent to adjusting the solution or encoding of the media document.

Growing Transformer APIs

As a part of the method to introduce new APIs, our engineering crew works intently with Google apps equivalent to Google Footage to check and experiment the brand new APIs. Experimental flags are first offered to permit efficiency enhancements. As soon as the consequences are a hit and conclusive, those experimental options are then constructed into the default API implementations or promoted to public APIs for all apps to make use of. This manner lets in Transformer APIs to be examined on all kinds of gadgets.

Transformer Adoption in apps

Apps which have been the usage of Transformer in manufacturing noticed in-app efficiency enhancements, much less code to handle, and higher developer revel in. Let’s take a better have a look at how Transformer has helped apps for his or her media-editing use circumstances.

Considered one of customers’ favourite options in Google Footage is reminiscence sharing, the place snippets of your lifestyles tale which can be curated and offered as Google Footage reminiscences can now be shared as movies to social media and chat apps. On the other hand, the method of mixing media pieces to create a video on software is useful resource extensive and matter to important latency, particularly on low-end gadgets. To cut back this latency and permit the function on a much wider vary of gadgets, Footage followed Transformer of their media advent pipeline. In conjunction with different enhancements made, the crew discovered that Transformer performed an element in lowering the median consumer latency for developing reminiscence movies by means of 41% on high-end gadgets and 27% on mid-range gadgets.

The Footage app additionally allows customers to accomplish media edits equivalent to trimming or rotating a video. Through adopting Transformer APIs for rotating movies, median save latency used to be lowered by means of 79% for acceptable movies. The app additionally followed Transformer’s API for optimizing video trimming, and noticed video save latency lower by means of 64%.

1 2nd On a regular basis is a non-public video magazine that is helping you create charming montages and timelapses. Probably the most app’s primary consumer trips is sequentially combining quick movies to create a significant film. After adopting Transformer for this use case, the app noticed that video encoding efficiency used to be as much as 5x quicker, letting them discover enabling 4k and HDR beef up. The Transformer adoption additionally helped lower related code by means of 30%, making it more straightforward for the builders to handle the code base.

BandLab is the next-generation song advent platform utilized by thousands and thousands world wide to make and percentage their song. The app in the beginning used MediaCodecs for his or her video advent use circumstances, however discovered that the low stage implementation led to local crashes that had been tough to debug. After researching extra on Transformer, the crew made the verdict emigrate from MediaCodecs to Transformer. General, it handiest took the crew 12 running days for the migration, and this led to a more effective codebase and extra maintainable pipeline for his or her media advent use circumstances. As well as, the app noticed that each one prior to now noticed local crashes had been not happening anymore.

What’s subsequent for Transformers?

We’re excited to look Transformer’s adoption within the developer group, and can proceed including new options to beef up extra media-editing use circumstances for the Android ecosystem together with:

• Higher beef up for previewing media edits
• Making improvements to the efficiency and developer revel in for video body extraction
• More uncomplicated integration with AI results
• and a lot more

Stay a watch out on what we’re running on within the Media3 Github, and document function requests to lend a hand form the way forward for Transformer!