Supporting Picture in Picture (PIP) in iOS application

Introduction

iOS 9 had brought some multitasking enhancements mainly for iPads. One of which is Picture in Picture (PIP) mode for video playback. In short, PIP allows user to leave the application and continue to watch a video in a floating resizable window. Also, the user can continue to use its iPad normally, without interrupting a playback. Overall, an enhanced multitasking!

IMG_0443.jpg
Picture in Picture mode in iOS

An iOS application can use either AVPlayerViewController or AVPlayerLayer to present a video content. The difference is, AVPlayerViewController provides a standard iOS player interface with the system provided playback controls, as shown in the image below, whereas AVPlayerLayer just presents a video and let application provide a custom interface for the playback controls.

IMG_0441.PNG
Standard iOS player interface in iOS 11

Configuring your application for PIP support

Configuring PIP in your application is a simple process. Make sure –

  1.  Build Settings->Architectures->Base SDK is set to either Latest iOS or >iOS 9.0.
  2. Capabilities->Background Modes->Audio, Airplay, and Picture in Picture is selected.
  3. Add UIBackgroundModes key in your Info.plist and set its value to audio. This lets iOS know that your application wants to use audio output even in the background mode. Additionally, your application needs to set a  AVAudioSessionCategoryPlayback category with an appropriate mode as follows –
// Access the shared, singleton audio session instance
let session = AVAudioSession.sharedInstance()
do {
    // Configure the audio session for movie playback
    try session.setCategory(AVAudioSessionCategoryPlayback,
                            mode: AVAudioSessionModeMoviePlayback,
                            options: [])
} catch let error as NSError {
    print("Failed to set the audio session category and mode: \(error.localizedDescription)")
    return
}


//Activate the session with the set category.
//Note from Apple -
//"You can activate the audio session at any time after setting its category,
//but it’s generally preferable to defer this call until your app begins audio playback.
//Deferring the call ensures that you won’t prematurely interrupt any other background audio
//that may be in progress."
do {
    try session.setActive(true)
} catch let error as NSError {
    print("Failed to activate the audio session with the set category: \(error.localizedDescription)")
    return
}

Here, we are accessing the shared AVAudioSession instance and setting a category named AVAudioSessionCategoryPlayback with the AVAudioSessionModeMoviePlayback mode, considering our application will play movies in the PIP mode. This helps iOS to make voice signals optimized for your playback. Once the category is set, we need to activate the audio session. Apple suggests to defer from activating until your application begins playback.

PIP invocation

PIP should always be user controlled and hence your application should not invoke PIP without user’s intention, especially when user is using the application. Apple recommends providing an option for your player to invoke PIP. If you’re using AVPlayerViewController to present a video playback, the standard iOS player interface automatically provides a button to invoke PIP. Whereas, if you have a custom interface for video controls, it’s your responsibility to provide an invocation point for PIP.

As an add-on, you can use native set of PIP images on the button so that user easily recognizes the PIP invocation point.  AVPictureInPictureViewController provides a class level interface to get the native images – pictureInPictureButtonStartImage(compatibleWith:) and  pictureInPictureButtonStopImage(compatibleWith:)

PIP implementation

AVPlayerViewController automatically manages PIP, provided your app is configured as mentioned above. Though, to work with AVPlayerLayer, AVKit provides AVPictureInPictureViewController class to help manage the video in PIP mode. Let’s see how you can make use of AVPictureInPictureViewController to support PIP in your application –

func beginPicutreInPictureMode() {
    //#1 - create a file url for a local video content
    let url = URL(fileURLWithPath: "video.mp4")
    
    //#2 - initialize AVPlayer instance with the url
    let player = AVPlayer(url: url)
    
    //#3 - initialize AVPlayerLayer instance with the AVPlayer
    let playerLayer = AVPlayerLayer(player: player)
    
    //#4 - Check if Picture in Picture mode is supported on user's device
    //Picture in Picture is only available on iPads with >iOS 9
    guard AVPictureInPictureController.isPictureInPictureSupported() else {
        print("Picture in Picture mode is not supported")
        return
    }
    
    //#5 - Initialize an instance of AVPictureInPictureController with the AVPlayerLayer instance
    //so that the video content displayed using AVPlayerLayer can be presented in PIP mode
    if let pipController = AVPictureInPictureController(playerLayer: layer) {
        //Assign self as a delegate to receive PIP state callbacks
        pipController.delegate = self
        //#6 - check whether PIP is possible at the current point in time
        //If PIP is not possible, we should not go ahead with the playback
        if pipController.isPictureInPicturePossible {
            //#7 - Video can be played in PIP mode.
            //Finally start PIP playback
            pipController.startPictureInPicture()
        } else {
            //#8 - isPictureInPicturePossible is a KVO enabled property
            //observing here for this property so that our class will be
            //notified when the PIP mode playback is actually possible.
            pipController.addObserver(pipController, forKeyPath: "isPictureInPicturePossible", options: [.new], context: nil)
        }
    }
}

override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
    guard keyPath == "isPictureInPicturePossible" else {
        return
    }
    
    //#9 read the KVO notification for the property isPictureInPicturePossible
    if let pipController = object as? AVPictureInPictureController {
        if pipController.isPictureInPicturePossible {
            //Video can be played in PIP mode.
            pipController.startPictureInPicture()
        }
    }
}

I hope the code and comments are self-explanatory. I would emphasize to make note of Step #4 and #6, as they are the necessary checks in the PIP workflow. Once all the verifications are done, startPictureInPicture() takes care of presenting a video in the PIP mode. Similarly, stopPictureInPicture() stops and exits out of the PIP mode.

Finally, implement the AVPictureInPictureControllerDelegate to manage the states of the PIP video playback.

extension ViewController: AVPictureInPictureControllerDelegate {
    
    func picture(_ pictureInPictureController: AVPictureInPictureController, restoreUserInterfaceForPictureInPictureStopWithCompletionHandler completionHandler: @escaping (Bool) -> Void) {
        //Update video controls of main player to reflect the current state of the video playback.
        //You may want to update the video scrubber position.
    }
    
    func pictureInPictureControllerWillStartPictureInPicture(_ pictureInPictureController: AVPictureInPictureController) {
        //Handle PIP will start event
    }
    
    func pictureInPictureControllerDidStartPictureInPicture(_ pictureInPictureController: AVPictureInPictureController) {
        //Handle PIP did start event
    }
    
    func picture(_ pictureInPictureController: AVPictureInPictureController, failedToStartPictureInPictureWithError error: Error) {
        //Handle PIP failed to start event
    }
    
    func pictureInPictureControllerWillStopPictureInPicture(_ pictureInPictureController: AVPictureInPictureController) {
        //Handle PIP will stop event
    }
    
    func pictureInPictureControllerDidStopPictureInPicture(_ pictureInPictureController: AVPictureInPictureController) {
        //Handle PIP did start event
    }
}

Application continue to run in a background mode when PIP is in progress. Consuming too many resources in background mode may cause your application to terminate. Hence, make sure to use minimal resources for seamless video playback.

I hope you manage to play video in PIP mode after this tutorial. Thank you for reading.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: