safari remote playback api

New WebKit Features in Safari 13.1

Apr 3, 2020

by Jon Davis

This year’s spring releases of Safari 13.1 for macOS Catalina, iPadOS, iOS, and watchOS bring a tremendous number of WebKit improvements for the web across Apple’s platforms. All of this with many more updates for improved privacy, performance, and a host of new tools for web developers.

Here’s a quick look at the new WebKit enhancements available with these releases.

Pointer and Mouse Events on iPadOS

The latest iPadOS 13.4 brings desktop-class pointer and mouse event support to Safari and WebKit. To ensure the best experience, web developers can use feature detection and adopt Pointer Events. Since a mouse or trackpad won’t send touch events, web content should not depend on touch events. Pointer Events will specify whether a mouse/trackpad or touch generated the event.

Web Animations API

These releases ship with support for the Web Animations API, a web standard offering developers a JavaScript API to create, query, and control animations, including direct control of CSS Animations and CSS Transitions. It offers a convenient unified model for programmatic animations, CSS Animations and Transitions. They can all now be directly controlled to pause, resume, seek, or change speed and direction, with less manual calculation. In addition, Web Inspector has been updated to show entries for them in the Media and Animations timeline.

Read more about Web Animations in WebKit and Web Animations in Safari 13.1 .

Async Clipboard API

WebKit brings the Async Clipboard API to this release of Safari. It provides access to the system clipboard and clipboard operations while keeping the webpage responsive. This API is much more flexible than DataTransfer, allowing developers to write multiple items with multiple types per item. Additionally, it brings programmatic paste to all websites on macOS and iOS.

The implementation is available through the navigator.clipboard API which must be called within user gesture event handlers like pointerdown or pointerup , and only works for content served in a secure context (e.g. https:// ). Instead of a permissions-based model for reading from the clipboard, a native UI is displayed when the page calls into the clipboard API; the clipboard can only be accessed if the user then explicitly interacts with the platform UI.

For more details see the original API specifications .

JavaScript Improvements

These releases include new JavaScript support for the replaceAll() method for strings and the new nullish coalescing operator ( ?? ).

The String.prototype.replaceAll() method does exactly what it suggests, replacing all occurrences of a given value in the string with a replacement string.

Learn more from the String.prototype.replaceAll Proposal.

The nullish coalesing operator ( ?? ) is a new operator that only evaluates and returns the expression on the right of the ?? if the result of the expression on the left of the ?? is null or undefined .

For more details see the Nullish Coalescing for JavaScript proposal .

ResizeObserver

The addition of ResizeObserver in WebKit enables developers to design components that are responsive to the container instead of just the viewport. This allows more flexible responsive designs, where containers can react to window size changes, orientation changes, and additions of new elements to the layout. The JavaScript API avoids the circular dependencies of trying to use media queries for element sizes in CSS. ResizeObserver addresses the problem by providing a means to observe changes in the layout size of elements.

For more read about ResizeObserver in WebKit .

HTML enterkeyhint Attribute

On iOS, WebKit supports the enterkeyhint attribute that allows a content author to provide a label for the enter key on virtual keyboards with values for: enter , done , go , next , previous , search , and send .

See the HTML Standard for more information.

CSS Shadow Parts

New support for CSS Shadow Parts allows web authors to style parts of web components without the need to understand how they are constructed. This provides a mechanism to define author-defined style parts akin to input element’s pseudo elements in WebKit.

See the CSS Shadow Parts specification for more information.

More CSS Additions

There are a number of new CSS additions in WebKit. New font keywords are available for using platform-specific fonts including ui-serif , ui-sans-serif , ui-monospace, and ui-rounded . WebKit also supports the line-break: anywhere value that adds a soft wrap opportunity around each character unit, including around any punctuation or preserved white spaces, in the middle of words, even ignoring limits against line breaks. Finally, WebKit includes support for the dynamic-range media query allowing authors to create styles specific to display capabilities.

Safari was the first to ship a picture-in-picture feature and has long supported the ability to specify a playback target for AirPlay. Safari for iOS and macOS now supports the standardizations of these features with the Picture-in-Picture API and Remote Playback API . There is also new support for HLS date-range metadata in DataCue .

Subtitles and Captions

WebKit is introducing enhancements to TextTrackCue for programmatic subtitle and caption presentation. This enables video publishers to continue storing captions in legacy or custom formats, and deliver them programmatically and still maintain the ability for users to control the presence and style of captions with system accessibility settings.

For more detail, see the WebKit TextTracks Explainer .

WebRTC Legacy Audio and Proxy Support

WebRTC support in WebKit has been updated so it can work in more places, with more systems. Support for DTMF allows WebKit to interact with legacy audio services. WebRTC Proxy support allows WebRTC to work in enterprise networks where firewalls may forbid UDP and require TCP to go through a dedicated proxy.

Performance Improvements

WebKit continues to deliver performance gains on benchmarks in these releases while also optimizing memory use. This release includes an 8-10% improvement on the Jetstream 2 benchmark. JavaScript Promises in particular showed a 2× improvement in the async-fs benchmark on JetStream 2. IndexedDB showed an improvement of 1.3× to 5× faster than before for most operations. There’s also faster Service Worker startup and more efficient CSS media query updates. Improved back-forward responsiveness helps history navigations feel snappier. Plus, a new Web Assembly interpreter dramatically improves startup time by around 8× for large WASM apps.

Security Improvements

WebKit has continued to harden security by fixing a number of bugs found through a process known as fuzzing. Following our announcement of deprecating TLS 1.0 and TLS 1.1 connections, this release now adds a “Not Secure” warning when connecting to a site where any resource is using either of these deprecated encryption protocols.

Intelligent Tracking Prevention Updates

There are several new enhancements to Intelligent Tracking Prevention including full third-party cookie blocking, cross-site document.referrers downgraded to their origins, and an expiry on non-cookie website data after seven days of Safari use and no user interaction on the website.

Read the “ Full Third-Party Cookie Blocking and More ” blog post for details.

Web Platform Quality Improvements

Areas of improved standards compliance and browser interoperability include more compatible gradient and position parsing, color component rounding, new support for the Q unit, and better calc() computed styles.

Web Inspector Updates

Web Inspector in Safari 13.1 includes new debugging experiences and adds several new tools to help web developers test functionality or identify issues.

Sources Tab

A new Sources Tab combines the Resources Tab and Debugger Tab into a single view, keeping more critical information in one place without the need to switch back and forth. Among the improvements, it includes improved support for debugging workers and has new JavaScript breakpoints , such as pausing on All Events or on All Microtasks .

Also new in the Sources Tab, developers can create use the “+” button in the lower left of the navigation sidebar to add an Inspector Bootstrap Script or Local Override . The Inspector Bootstrap Scripts is a snippet of JavaScript that is guaranteed to be the first script evaluated after any new global object is created in any page or sub-frame, regardless of URL, so long as Web Inspector is open. A Local Override can be added to override any resource loaded into the page, giving developers the ability to change files and preview those the changes on pages that they might ordinarily not be able to change.

Both the Sources Tab and the Network Tab also benefit from improved displaying of HTML and XML content, including being able to pretty print or viewing any request/response data as a simulated DOM tree.

The Layers Tab is also newly available in this release. It provides a 3D visualization and complete list of the rendering layers used to display the page. It also includes information like layer count and the memory cost of all the layers, both of which can help point developers to potential performance problems.

Read the “ Visualizing Layers in Web Inspector ” blog post for details.

Script Blackboxing

Script Blackboxing is another powerful tool, focused on helping developers debug behaviors built on top of a JavaScript library or framework. By setting up a blackbox for any library or framework script, the debugger will ignore any pauses that would have happened in that script, instead deferring the pause until JavaScript execution continues to a statement outside of the blackboxed script.

Redesigned Color Picker

Other additions to Web Inspector give content authors more insight for design and user experience. A redesigned color picker uses a square design for more precise color selection and includes support for wide-gamut colors with a white guide line that shows the edge of sRGB to Display-P3 color space.

Learn more from the “ Wide Gamut Color in CSS with Display-P3 ” blog post.

Customized AR QuickLook

• Apple Pay button styles
• Action button label
• An item title
• Item subtitle

Or, authors can create an entirely custom banner with HTML:

For more information, read about Adding an Apple Pay Button or a Custom Action in AR Quick Look .

These improvements are available to users running watchOS 6.2, iOS and iPadOS 13.4, macOS Catalina 10.15.4, macOS Mojave 10.14.6 and macOS High Sierra 10.13.6. These features were also available to web developers with Safari Technology Preview releases. Changes in this release of Safari were included in the following Safari Technology Preview releases: 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 . Download the latest Safari Technology Preview release to stay on the forefront of future web platform and Web Inspector features. You can also use the WebKit Feature Status page to watch for changes to your favorite web platform features.

Send a tweet to @webkit or @jonathandavis to share your thoughts on this release.. If you run into any issues, we welcome your bug reports for Safari, or WebKit bugs for web content issues.

View in English

More Videos

Streaming is available in most browsers, and in the Developer app.

Coordinate media playback in Safari with Group Activities

Create SharePlay experiences that people can enjoy on the web and in your companion app. Learn how you can use the Group Activities framework in combination with a companion website to bring SharePlay to Safari, letting people connect with each other for enjoyable group interactions — even if they haven't yet downloaded your app from the App Store.

• Group Activities
• Supporting Coordinated Media Playback

Related Videos

• Coordinate media experiences with Group Activities
• Design for Group Activities
• Develop advanced web content
• Meet Group Activities

♪ Bass music playing ♪ ♪ Jer Noble: Hello, I'm Jer! And I work on media playback in WebKit and Safari here at Apple. People love watching their favorite shows and movies and listening to their music together, whether it’s with people in the same room, next door, or on the other side of the world. And now with GroupActivities, it’s easier than ever to provide a SharePlay experience to your users, wherever they are. In this session, we'll cover four topics. An introduction to coordinated playback in Safari; how to prepare your iOS app to work with SharePlay in Safari; how to adopt the Media Session API in your app's web page; and a new web API for coordinating playback across multiple devices. So let’s start by talking a little bit about SharePlay and Safari. In the previous session, you learned how to create an amazing SharePlay experience in your iPhone and iPad applications with GroupActivities. And on macOS Monterey, you can bring that same experience -- along with your great iPhone and iPad apps -- to Mac using Catalyst. And if your application has a companion website, you can bring that same amazing SharePlay experience to your Mac Safari 15 users, without requiring those users to download your app from the App Store. And that’s what I’m going to show you how to do in this session. I’ve been working on an iPhone application that lets you watch movie trailers. I’ve already added support for GroupActivities to my app, and that lets friends watch movie trailers together over FaceTime. But my movie trailers app also has a website. And here’s how it should look once I add support for GroupActivities in that web page as well. I’m on a FaceTime call with my friend Sam. Sam loves watching movie trailers, so I’ve asked them to help me test this new Movie Trailers app. When they start playing a new trailer in that app during our call, I’ll be invited to watch that trailer through a notification. It will give me the option to join the session in Safari. When I click that notification, Safari will be launched with a URL for the shared content, and when Sam starts playing the trailer, it will play here in Safari at the same time. I can pause, play, and even seek the video here in Safari, and everyone else in the session will see those same commands reflected in their local video at the same time. It's going to be really great! So, what will it take to add GroupActivities support to the movie trailers website? Before we take a look under the hood of Safari -- at the new Web APIs available in Safari 15 -- let's talk about how to get your app ready to work with Safari through GroupActivities. In order to join a GroupActivities session with Safari, a user with your iPhone or iPad app will need to create a GroupSessionActivity with a fallbackURL that points to your website. The fallbackURL must identify not just your website, but the specific content to be played. When a Watch Together invitation is issued by an iPhone user to a user on a Mac, Safari will be launched and asked to load the URL that you provide in GroupSessionActivity. Once the fallbackURL is loaded in Safari, your site will need to implement a few simple but important Web APIs. The first is Media Session. Media Session is a standard Web API which allows your site to tell the browser -- Safari -- more about the current state of media playback within your page. Your site can tell the browser the current play state, the duration of your content. It can provide artwork for the current playback item. It can even tell the browser that the current content is an advertisement that can be skipped. Once the Browser has all this metadata available, it can provide that metadata to the rest of the operating system and present it to your users outside of the browser window itself. One place the operating system displays that data is within something called "Now Playing." Now Playing is a great way for users to quickly see and control the media that they’re enjoying right now. In iOS, users can access Now Playing from the Control Center as well as from the Lock screen. And now in macOS Monterey, users can access Now Playing directly from the menu bar. And Safari will use the information provided through Media Session to populate Now Playing with your content’s title, duration, play state, artwork, and more. Now Playing also allows users to control playback of what they’re listening to, by showing the user a simple set of playback controls. Your site can register with the Media Session for specific actions to perform when a user interacts with Now Playing, so that for example, when a user taps the Play button in Now Playing, Safari can run your site’s own action handler to begin playback. Media Session gives your site complete control over what metadata to expose to the browser, as well as control over how to implement playback commands. Need to do some set up or preflighting before beginning playback? No problem. With mediaSession’s play action handler, now you can. If your site has previously adopted Media Session, you’re halfway done already. But the movie trailers site hasn’t adopted Media Session yet, so let’s add support for that now. This site will want to add actions for play, pause, and seekto. To do so, we’ll use the mediaSession property of navigator. Let’s first add an action handler for play. We’ll call mediaSession. setActionHandler(), passing in the string play and an inline function that just calls play() on our video element. Then, we’ll do the same for pause and seekto. And to make sure that we’re accurately representing playback state to the browser, we’ll add event listeners to our video element and update mediaSession when that state changes. First, let’s add a function called updateMediaSessionState(). It will query the video element to find out if it’s playing or paused. Then it will set mediaSession’s playbackState property to either the string playing or paused, depending on the video element’s playback state. Next, it will query the video element’s duration, playback rate, and current time. Then, using those values, it will call mediaSession’s setPositionState() function, passing in those values in the dictionary parameter. Then, let’s add an event listener to detect when those values change, and call updateMediaSessionState() to update mediaSession with those new values. We’ll need an event listener for playing, pause, duration change, rate change, and time change. And last, because the movie trailers website has both a title string and a URL for the title artwork, we can set the Media Session’s metadata property to a new MediaMetadata object containing those values. So let’s try it out! I’ll reload my page in Safari, begin playing, then click on the Now Playing icon in the system menu. And I see the title and artwork in the Now Playing panel. And when I tap the Pause button, our pause action handler is called and playback pauses. Media Session provides the building blocks for a Watch Together experience in a web page, by providing a mechanism for the browser to tell the page it should start or stop playback. Now we’re ready move onto the second part of creating a cowatching experience in our web page: coordinating playback across every device in the session. To do so, we’ve extended Media Session by adding a new property on it called "Coordinator." Your page will use Media Session's Coordinator to inform every other user in the session of the page’s intent to begin or pause playback, to seek to a specific point in the media’s timeline, or to move to the next item in a playlist. The Coordinator will communicate those intents with all the other devices in the session and listen for changes to every other device's play state, resolve conflicts, and wait for everyone to be ready to begin playback. And when everyone is ready, the Coordinator will use the Media Session action handlers we added earlier to start playback along all the other users in the session. And if another user pauses playback for whatever reason, the Coordinator will call your Media Session’s pause action handler. That said, I’d like to point out a few caveats. The Coordinator interface is an experimental web API; it has yet to go through standardization, and while that process will likely improve this API, some aspects of the API may change. To find out more, please watch the W3C’s Media Session GitHub page. This API is currently only implemented in Safari, though the GroupActivities framework, which Safari uses to implement the Coordinator, can -- and we hope will -- be adopted by other Mac browsers. And the final caveat is that while your users can join an existing session in Safari, a GroupSession currently must be initiated from an iPhone, iPad, or macOS application. So let’s add support for the Coordinator now. The first thing I’d like to do is join the session once it becomes available. So I’m going to add a coordinatorchange event listener to mediaSession. In the handler, if there’s a non-null coordinator property on mediaSession, I’ll call its join() function. I’d also like to add an icon to my controls to show that we’re playing back in a session. So in the handler, I’ll either show or hide the icon, based on whether there’s a coordinator property on mediaSession. I’m also going to hook up a click event listener to the icon to call leave() on the coordinator. Next, I’m going to modify the event handlers for my custom controls to check for the presence of a coordinator, and if we have one, call the appropriate coordinator function instead. I’ll add one for my Play button, and then for my Pause button, and also for timeline slider. And let’s see it in action! I’ve got an invite here from my friend Sam, and I’m going to accept it. My page loads to the same content that Sam is watching on their iPhone. And when I click the Play button, both Sam's iPhone and my web page begin playing simultaneously. Allie Fox: The way we get there is an adventure. Dina: I'm scared. Allie: If it didn't feel that way, it wouldn't be an adventure. ♪ That was great. Now that the trailer's ended, Sam uses the iPhone to scrub back to the beginning.

We think that users are going to love watching or listening to their favorite content together in your applications, whether it’s on their iPhone, iPad, Apple TV, or in Safari running on their Mac. And we hope you can now bring that experience to all of your users, wherever they are. For more information about GroupActivities APIs, start with the sessions "Coordinate media playback with Group Activities" and "Design for Group Activities." If you’re interested in the other new web APIs that will be available in Safari 15, make sure to check out the session, "Develop advanced web content." Thanks for coming and enjoy the rest of your WWDC21. ♪

2:50 - Preparing your app

5:29 - Adopting Media Session

9:32 - Adopting Coordinator

Looking for something specific? Enter a topic above and jump straight to the good stuff.

An error occurred when submitting your query. Please check your Internet connection and try again.

Picture-in-Picture

Editor’s Draft , 2 September 2024

Copyright © 2024 World Wide Web Consortium . W3C ® liability , trademark and permissive document license rules apply.

This specification provides APIs to allow websites to create a floating video window always on top of other windows so that users may continue consuming media while they interact with other content sites, or applications on their device.

Status of this document

This section describes the status of this document at the time of its publication. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at https://www.w3.org/TR/.

Feedback and comments on this specification are welcome. GitHub Issues are preferred for discussion on this specification. Alternatively, you can send comments to the Media Working Group’s mailing-list, [email protected] ( archives ). This draft highlights some of the pending issues that are still to be discussed in the working group. No decision has been taken on the outcome of these issues including whether they are valid.

This document was published by the Media Working Group as an Editor’s Draft. This document is intended to become a W3C Recommendation.

Publication as an Editor’s Draft does not imply endorsement by W3C and its Members.

This document was produced by a group operating under the W3C Patent Policy . W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy .

This document is governed by the 03 November 2023 W3C Process Document .

1. Introduction

This section is non-normative.

Many users want to continue consuming media while they interact with other content, sites, or applications on their device. A common UI affordance for this type of activity is Picture-in-Picture (PiP), where the video is contained in a separate miniature window that is always on top of other windows. This window stays visible even when the user agent is not visible. Picture-in-Picture is a common platform-level feature among desktop and mobile OSs.

This specification extends HTMLVideoElement allowing websites to initiate and control this behavior by exposing the following sets of properties:

Notify the website when it enters and leaves Picture-in-Picture mode.

Allow the website to trigger Picture-in-Picture mode via a user gesture on a video element.

Allow the website to know the size of the Picture-in-Picture window and notify the website when it changes.

Allow the website to exit Picture-in-Picture mode.

Allow the website to check if Picture-in-Picture mode can be triggered.

2. Examples

2.1. add a custom picture-in-picture button, 2.2. monitor video picture-in-picture changes, 2.3. update video size based on picture-in-picture window size changes, 3. concepts, 3.1. internal slot definitions.

A user agent has:

An initiators of active Picture-in-Picture sessions list of zero or more origins , which is initially empty.

Note: In case a user agent supports multiple Picture-in-Picture windows, the list allows duplicates.

An origin is said to have an active Picture-in-Picture session if any of the origins in initiators of active Picture-in-Picture sessions are same origin-domain with origin.

3.2. Request Picture-in-Picture

When the request Picture-in-Picture algorithm with video is invoked, the user agent MUST run the following steps:

If Picture-in-Picture support is false , throw a NotSupportedError and abort these steps.

If the document is not allowed to use the policy-controlled feature named "picture-in-picture" , throw a SecurityError and abort these steps.

If video ’s readyState attribute is HAVE_NOTHING , throw a InvalidStateError and abort these steps.

If video has no video track, throw a InvalidStateError and abort these steps.

If video ’s disablePictureInPicture is true, the user agent MAY throw an InvalidStateError and abort these steps.

If pictureInPictureElement is null and the relevant global object of this does not have transient activation , throw a NotAllowedError and abort these steps.

If video is pictureInPictureElement , abort these steps.

Set pictureInPictureElement to video .

Let Picture-in-Picture window be a new instance of PictureInPictureWindow associated with pictureInPictureElement .

Append relevant settings object ’s origin to initiators of active Picture-in-Picture sessions .

Queue a task to fire an event named enterpictureinpicture using PictureInPictureEvent at the video with its bubbles attribute initialized to true and its pictureInPictureWindow attribute initialized to Picture-in-Picture window .

If pictureInPictureElement is fullscreenElement , it is RECOMMENDED to exit fullscreen .

It is RECOMMENDED that video frames are not rendered in the page and in the Picture-in-Picture window at the same time but if they are, they MUST be kept in sync.

When a video is played in Picture-in-Picture mode, the states SHOULD transition as if it was played inline. That means that the events SHOULD fire at the same time, calling methods SHOULD have the same behaviour, etc. However, the user agent MAY transition out of Picture-in-Picture when the video element enters a state that is considered not compatible with Picture-in-Picture.

Styles applied to video (such as opacity, visibility, transform, etc.) MUST NOT apply in the Picture-in-Picture window. Its aspect ratio is based on the video size.

It is also RECOMMENDED that the Picture-in-Picture window has a maximum and minimum size. For example, it could be restricted to be between a quarter and a half of one dimension of the screen.

3.3. Exit Picture-in-Picture

When the exit Picture-in-Picture algorithm is invoked, the user agent MUST run the following steps:

If pictureInPictureElement is null , throw a InvalidStateError and abort these steps.

Run the close window algorithm with the Picture-in-Picture window associated with pictureInPictureElement .

Queue a task to fire an event named leavepictureinpicture using PictureInPictureEvent at the video with its bubbles attribute initialized to true and its pictureInPictureWindow attribute initialized to Picture-in-Picture window associated with pictureInPictureElement .

Unset pictureInPictureElement .

Remove one item matching relevant settings object ’s origin from initiators of active Picture-in-Picture sessions .

It is NOT RECOMMENDED that the video playback state changes when the exit Picture-in-Picture algorithm is invoked. The website SHOULD be in control of the experience if it is website initiated. However, the user agent MAY expose Picture-in-Picture window controls that change video playback state (e.g., pause).

As one of the unloading document cleanup steps , run the exit Picture-in-Picture algorithm .

3.4. Disable Picture-in-Picture

Some pages may want to disable Picture-in-Picture mode for a video element; for example, they may want to prevent the user agent from suggesting a Picture-in-Picture context menu in some cases. To support these use cases, a new disablePictureInPicture attribute is added to the list of content attributes for video elements.

The disablePictureInPicture IDL attribute MUST reflect the content attribute of the same name.

If the disablePictureInPicture attribute is present on the video element, the user agent MAY prevent the video element from playing in Picture-in-Picture mode or present any UI to do so.

When the disablePictureInPicture attribute is added to a video element, the user agent MAY run these steps:

Reject any pending promises returned by the requestPictureInPicture() method with InvalidStateError .

If video is pictureInPictureElement , run the exit Picture-in-Picture algorithm .

3.5. Interaction with Fullscreen

It is RECOMMENDED to run the exit Picture-in-Picture algorithm when the pictureInPictureElement fullscreen flag is set.

3.6. Interaction with Remote Playback

The [Remote-Playback] specification defines a local playback device and a local playback state . For the purpose of Picture-in-Picture, the playback is local and regardless of whether it is played in page or in Picture-in-Picture.

3.7. Interaction with Media Session

The API will have to be used with the [MediaSession] API for customizing the available controls on the Picture-in-Picture window.

3.8. Interaction with Page Visibility

When pictureInPictureElement is set, the Picture-in-Picture window MUST be visible, even when the Document is not in focus or hidden. The user agent SHOULD provide a way for users to manually close the Picture-in-Picture window.

The Picture-in-Picture window visibility MUST NOT be taken into account by the user agent to determine if the system visibility state of a traversable navigable has changed.

3.9. One Picture-in-Picture window

Operating systems with a Picture-in-Picture API usually restrict Picture-in-Picture mode to only one window. Whether only one window is allowed in Picture-in-Picture mode will be left to the implementation and the platform. However, because of the one Picture-in-Picture window limitation, the specification assumes that a given Document can only have one Picture-in-Picture window.

What happens when there is a Picture-in-Picture request while a window is already in Picture-in-Picture will be left as an implementation detail: the current Picture-in-Picture window could be closed, the Picture-in-Picture request could be rejected or even two Picture-in-Picture windows could be created. Regardless, the User Agent will have to fire the appropriate events in order to notify the website of the Picture-in-Picture status changes.

4.1. Extensions to HTMLVideoElement

The requestPictureInPicture() method, when invoked, MUST return a new promise promise and run the following steps in parallel :

Let video be the video element on which the method was invoked.

Run the request Picture-in-Picture algorithm with video .

If the previous step threw an exception, reject promise with that exception and abort these steps.

Resolve promise with the Picture-in-Picture window associated with pictureInPictureElement .

4.2. Extensions to Document

The pictureInPictureEnabled attribute’s getter must return true if Picture-in-Picture support is true and the context object is allowed to use the feature indicated by attribute name picture-in-picture , and false otherwise.

Picture-in-Picture support is false if there’s a user preference that disables it or a platform limitation. It is true otherwise.

The exitPictureInPicture() method, when invoked, MUST return a new promise promise and run the following steps in parallel :

Run the exit Picture-in-Picture algorithm .

Resolve promise .

4.3. Extension to DocumentOrShadowRoot

The pictureInPictureElement attribute’s getter must run these steps:

If the context object is not connected , return null and abort these steps.

Let candidate be the result of retargeting Picture-in-Picture element against the context object .

If candidate and the context object are in the same tree , return candidate and abort these steps.

Return null .

4.4. Interface PictureInPictureWindow

A PictureInPictureWindow instance represents a Picture-in-Picture window associated with an HTMLVideoElement . When instantiated, an instance of PictureInPictureWindow has its state set to opened .

When the close window algorithm with an instance of PictureInPictureWindow is invoked, its state is set to closed .

The width attribute MUST return the width in CSS pixels of the Picture-in-Picture window associated with pictureInPictureElement if the state is opened . Otherwise, it MUST return 0.

The height attribute MUST return the height in CSS pixels of the Picture-in-Picture window associated with pictureInPictureElement if the state is opened . Otherwise, it MUST return 0.

When the size of the Picture-in-Picture window associated with pictureInPictureElement changes, the user agent MUST queue a task to fire an event named resize at pictureInPictureElement .

4.5. Event types

Fired on a HTMLVideoElement when it enters Picture-in-Picture.

Fired on a HTMLVideoElement when it leaves Picture-in-Picture mode.

Fired on a PictureInPictureWindow when it changes size.

4.6. Task source

The task source for all the tasks queued in this specification is the media element event task source of the video element in question.

4.7. CSS pseudo-class

The :picture-in-picture pseudo-class MUST match the Picture-in-Picture element. It is different from the pictureInPictureElement as it does NOT apply to the shadow host chain.

5. Security considerations

To limit potential abuse through spoofing, the API applies only to HTMLVideoElement . User interaction with the Picture-in-Picture window is intentionally limited so that the only effect is on the Picture-in-Picture window itself or the media being played.

5.1. Secure Context

The API is not limited to [SECURE-CONTEXTS] because it exposes a feature to web applications that user agents usually offer natively on all media regardless of the browsing context .

5.2. Permissions Policy

This specification defines a policy-controlled feature named "picture-in-picture" that controls whether the request Picture-in-Picture algorithm may return a SecurityError and whether pictureInPictureEnabled is true or false .

The default allowlist for this feature is * .

6. Acknowledgments

Thanks to Jennifer Apacible, Zouhir Chahoud, Marcos Cáceres, Philip Jägenstedt, Jeremy Jones, Chris Needham, Jer Noble, Justin Uberti, Yoav Weiss, and Eckhart Wörner for their contributions to this document.

Conformance

Document conventions.

Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this specification.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes. [RFC2119]

Examples in this specification are introduced with the words “for example” or are set apart from the normative text with class="example" , like this:

This is an example of an informative example.

Informative notes begin with the word “Note” and are set apart from the normative text with class="note" , like this:

Note, this is an informative note.

Conformant Algorithms

Requirements phrased in the imperative as part of algorithms (such as "strip any leading space characters" or "return false and abort these steps") are to be interpreted with the meaning of the key word ("must", "should", "may", etc) used in introducing the algorithm.

Conformance requirements phrased as algorithms or specific steps can be implemented in any manner, so long as the end result is equivalent. In particular, the algorithms defined in this specification are intended to be easy to understand and are not intended to be performant. Implementers are encouraged to optimize.

Terms defined by this specification

Terms defined by reference, normative references, informative references.

RemotePlayback

The RemotePlayback interface of the Remote Playback API allows the page to detect availability of remote playback devices, then connect to and control playing on these devices.

Instance properties

Represents the RemotePlayback connection's state. One of:

The user agent is attempting to initiate remote playback with the selected device.

The transition from local to remote playback has happened, all commands will now take place on the remote device.

The remote playback has not been initiated, has failed to initiate, or has been stopped.

Instance methods

A Promise that resolves with a callbackId of an available remote playback device.

Cancels the request to monitor the availability of remote playback devices.

Prompts the user to select and give permission to connect to a remote playback device.

Fired when the user agent initiates remote playback.

Fired when the user agent successfully connects to the remote device.

Fired when the user agent disconnects from the remote device.

The following example demonstrates a player with custom controls that support remote playback. Initially the button used to select a device is hidden:

The RemotePlayback.watchAvailability() method is used to watch for available remote playback devices. If a device is available, use the callback to show the button.

Specifications

Browser compatibility.

© 2005–2023 MDN contributors. Licensed under the Creative Commons Attribution-ShareAlike License v2.5 or later. https://developer.mozilla.org/en-US/docs/Web/API/RemotePlayback

• Skip to main content
• Skip to search
• Skip to select language
• Sign up for free

Cross-browser audio basics

This article provides:

• a basic guide to creating a cross-browser HTML audio player with all the associated attributes, properties, and events explained
• a guide to custom controls created using the Media API

Basic Audio example

The code below is an example of a basic audio implementation using HTML5:

Note: You can also use an MP4 file instead of MP3. MP4 files typically contain AAC encoded audio. You can use type="audio/mp4" . (Currently, browsers that support mp3 also support mp4 audio).

• Here we define an <audio> element with multiple sources — we do this as not all browsers support the same audio formats. To ensure reasonable coverage, we should specify at least two different formats. The two formats that will give maximum coverage are mp3 and ogg vorbis.
• src contains the path to the audio file to be loaded (relative or absolute).
• type is used to inform the browser of the file type. If omitted, most browsers will attempt to guess this from the file extension.
• If the <audio> element is not supported then <audio> and <source> will be ignored. However, any supported text or elements that you define within <audio> will be displayed or acted upon. So the ideal place to create a fallback or inform of incompatibility is before the closing </audio> tag. In this case, we've provided a simple paragraph including a link to download the audio directly.
• The controls attribute on the <audio> element is specified when we require the browser to provide us with default playback controls. If you don't specify this attribute, no controls will appear — and you will instead have to create your own controls and program their functionality using the Media API (see below). However, that can be a good approach, because the default controls look different among various browsers. So creating your own controls ensures a consistent look for the controls across all browsers.

HTML audio in detail

Now we've looked at a basic example, let's now explore the different aspects of HTML audio in more detail.

Audio HTML attributes

We can specify a number of attributes with the audio tag to further determine the way audio is initialized.

Specifying autoplay will cause the audio to start playing as soon as possible and without any user interaction — in short, the audio will autoplay.

Note: This value is often ignored on mobile platforms, and its use is not recommended unless really necessary. Autoplaying audio (and video) is usually really annoying. Plus browsers have policies that will block autoplay entirely in many situations. See the Autoplay guide for media and Web Audio APIs for details.

The loop attribute will ensure that upon getting to the end of the audio clip, the audio clip will loop back to the beginning and start playing again.

If you want the audio to start muted (no volume), add the muted attribute.

Note: This value is often ignored on mobile platforms.

The preload attribute allows you to specify a preference for how the browser preloads the audio, in other words, which part of the file it downloads when the <audio> element is initialized, and before the play button is pressed.

preload can take 3 different values:

• none : Don't download anything before the play button is pressed.
• metadata : Download the audio metadata; this is usually the best option, as it allows you to access and display information such as audio length, and allow the browser to work out which audio file it should use.
• auto : Download the whole audio file as soon as possible. This is generally not a good option unless you can guarantee your users will have a fast network connection.

We specify the controls attribute when we require the browser to provide us with its default playback controls.

As mentioned above, you can use the <source> element to specify one or more source audio files. Alternatively, you can include the src attribute directly on the <audio> element to specify a single source file.

As mentioned above, to be sure that the browser knows what type of file is being specified, it's good practice to specify a type attribute alongside the src attribute. The type attribute specifies the MIME type or Internet Media Type of the file.

Manipulating the Audio Element with JavaScript

In addition to being able to specify various attributes in HTML, the <audio> element comes complete with several properties and methods that you can manipulate via JavaScript.

Given the following HTML:

You can grab the <audio> element like this:

Alternatively, you can create a new element. Here's an example of creating an <audio> element, setting the media to play, playing and pausing, and then playing from 5 seconds into the audio:

Let's explore the available properties and methods in more detail.

The play() method is used to tell the audio to play. It takes no parameters.

The pause() method is used to tell the audio to pause. It takes no parameters.

Note: There is no stop method — to implement a stop function, you'd have to pause the media then set the currentTime property value to 0.

canPlayType

The canPlayType() method asks the browser whether a certain audio file type is supported. It takes the mime type of the type to check as a parameter.

canPlayType() returns one of three values:

• "" (an empty string)

In practice, we usually check if the result is true or false. Non-empty strings are true.

Note: A very early spec specified that the browser should return no instead of an empty string, but thankfully the number of people using older browsers that implement this version of the spec are few and far between.

currentTime

The currentTime property gets or sets the current time the audio should play at. This is useful in many ways, for example since play() does not take a parameter we need to set the point to play from separately if we don't want it to be 0.

The value of currentTime is a number which represents the time in seconds.

The volume property allows us to set the audio volume, as a number between 0 and 1.

Creating your own custom audio player

The JavaScript media API allows you to create your own custom player. Let's take a look at a very minimal example. We can combine HTML and JavaScript to create a very simple player with a play and a pause button. First, we'll set up the audio in the HTML, without the controls attribute, since we are creating our own controls:

Next, we attach some functionality to the player using JavaScript:

Media loading events

Above we have shown how you can create a very simple audio player, but what if we want to show progress, buffering and only activate the buttons when the media is ready to play? Fortunately, there are a number of events we can use to let our player know exactly what is happening.

First, let's take a look at the media loading process in order:

The loadstart event tells us that load process has started and the browser is connecting to the media.

durationchange

If you just want to know as soon as the duration of your media is established, this is the event for you. This can be useful because the initial value for duration is NaN (Not a Number), which you probably don't want to display to your users.

loadedmetadata

Metadata can consist of more than just duration — if you want to wait for all the metadata to download before doing something, you can detect the loadedmetadata event.

The loadeddata event is fired when the first bit of media arrives. The playhead is in position but not quite ready to play.

The progress event indicates that the download of media is still in progress. It is good practice to display some kind of 'loader' at this point.

canplay is a useful event to detect should you want to determine whether the media is ready to play. You could, for example, disable custom controls until this event occurs.

canplaythrough

canplaythrough is similar to canplay but it lets you know that the media is ready to be played all the way through (that is to say that the file has completely downloaded, or it is estimated that it will download in time so that buffering stops do not occur).

Media loading event order

To recap, the order of the media loading events are:

loadstart > durationchange > loadedmetadata > loadeddata > progress > canplay > canplaythrough

Loading interruption events

We also have a few events available that will fire when there is some kind of interruption to the media loading process.

Media data is no longer being fetched even though the file has not been entirely downloaded.

Media data download has been aborted but not due to an error.

An error is encountered while media data is being downloaded.

The media buffer has been emptied, possibly due to an error or because the load() method was invoked to reload it.

Media data is unexpectedly no longer available.

Media Playing Events

We also have another set of events that are useful for reacting to the state of the media playback.

The timeupdate event is triggered every time the currentTime property changes. In practice, this occurs every 250 milliseconds. This event can be used to trigger the displaying of playback progress.

The playing event is initiated when playback is ready to start after having being paused due to lack of media data.

The waiting event is triggered when playback has stopped due to lack of media data, although it is expected to resume once data becomes available.

The play event is initiated after the play() method is returned or when the autoplay attribute has caused playback to begin. This is when the state of the media switches from paused to playing.

The pause event is triggered after the pause() method is returned. This is when the states switch from playing to paused.

The ended event is initiated when the end of the media is reached.

volumechange

The volumechange event signifies that the volume has changed; that includes being muted.

An Audio Player with Feedback

Consider this snippet of HTML:

Styled like so:

Now let's wire this thing up with JavaScript:

You should end up with something like this:

Seeking using the seek bar

This is a good start, but it would be nice to be able to navigate the audio using the progress bar. Fortunately this isn't too difficult to implement.

First of all, we apply a quick update to the progress bar CSS to display the hand pointer on hover:

Then we add the code that detects the click and moves the 'playhead' to the correct position:

Ok, we're getting there, but there is another piece of useful information we can display: the amount of audio has been buffered or downloaded in advance.

There are a couple of properties we haven't looked at yet, buffered and seekable .

This property lets us know which parts of the audio has been buffered (downloaded in advance). It returns something called a TimeRanges object.

The seekable property informs you of whether you can jump directly to that part of the media without further buffering.

Buffering events

There are also a couple of events related to buffering:

The seeking event is fired when media is being sought.

seeked occurs when the seeking attribute changes to false .

Note: You can read more on Buffering, Seeking and Time Ranges elsewhere.

• Buffering, Seeking and Time Ranges
• HTMLMediaElement events
• Event reference > Media
• Using HTML audio and video
• Creating a cross-browser video player
• jPlayer : An open source audio and video library for jQuery and Zepto.

Loading search index…

No recent searches

No results for " Query here "

• to navigate

Search by FlexSearch

Example Reference

Remote playback api.

The Remote Playback API extends the HTMLMediaElement to enable the control of remote playback of media from a webpage.

Concepts and Usage

Remote playback devices are connected devices such as TVs, projectors, or speakers. The API takes into account wired devices connected via HDMI or DVI, and wireless devices, for example Chromecast or AirPlay.

The API enables a page, which has an media element such as a video or audio file, to initiate and control playback of that media on a connected remote device. For example, playing a video on a connected TV.

Note: Safari for iOS has some APIs which enable remote playback on AirPlay. Details of these can be found in the Safari 9.0 release notes .

Android versions of Firefox and Chrome also contain some remote playback features. These devices will show a Cast button if there is a Cast device available in the local network.

Allows the page to detect availability of remote playback devices, then connect to and control playing on these devices.

The following example demonstrates a player with custom controls that supports remote playback. Initially the button used to select a device is hidden.

The RemotePlayback.watchAvailability() method watches for available remote playback devices. If a device is available, use the callback to show the button.

Specifications

Browser compatibility.

© 2005–2023 MDN contributors. Licensed under the Creative Commons Attribution-ShareAlike License v2.5 or later. https://developer.mozilla.org/en-US/docs/Web/API/Remote_Playback_API

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications You must be signed in to change notification settings

Remote Playback API #654

luwes May 27, 2023 Maintainer

Beta Was this translation helpful? Give feedback.

Replies: 2 comments

{{editor}}'s edit, bdougherty may 27, 2023, luwes dec 7, 2023 maintainer author.

• Numbered list
• Unordered list
• Attach files

Select a reply