Rendering GIFs as MP4s in iOS with Swift

Rendering GIFs as MP4s in iOS with Swift

At Barstool Sports we wanted to move away from using animated gifs and use MP4 video in their place. Why? I'll let the folks at Fastly explain that, here. This required work to be done in 2 separate ways in the Barstool Sports iOS app.

WKWebView

Our in-house CMS, Barstool HQ, delivers our content to the Barstool Sports iOS app in a WKWebView. Basically, HQ delivers HTML with video tags that are animated images. WKWebView does not support autoplaying these video tags automatically and a few changes need to be made. I could explain it but Thomas Visser explains it perfectly here. And he's the only reason I got it working in our app. The most important bit from his article that you'll want to pay attention to is this,

Make sure to change the configuration before passing it to WKWebView. I first tried to change the configuration through the webview (webView.configuration.allowsInlineMediaPlayback = true), but that silently failed.

UIImageView

UIImageView does not let you animate an MP4 file. We needed to use an AVPlayer for that. So we created a view that would have an UIImageView and an AVPlayer, with some logic to determine which to use. We need this because in our use case some images are normal and static while others are animated, and we don't know which until runtime.

In the code below you'll notice that we are using 2 third party frameworks, Nuke and Bluebird. We use Nuke for their excellent caching feature, and we are using Bluebird for its excellent implementation of Promises. (Side note.. Bluebird is an excellent framework developed by Barstool Sports Head of engineering Andrew Barba.) We've been using futures and promises in Swift for a long time here at Barstool Sports and we try to always use the latest that Swift has to offer. With that said, we are very excited to remove Bluebird from the codebase and replace it with async/await.

For those of you here who just want to see some code, here it is.

Code

To start we needed a view that could be either a UIImage or an AVPlayer. We built this view in code, but this could also be done with a Storyboard or Xib.

public class ImageOrMP4GifView: UIView {
    private var videoPlayer: AVPlayer?
    private var videoPlayerLayer: AVPlayerLayer?
    private var imageURL: URL?
    
    private lazy var image: UIImageView = {
        let image = UIImageView()
        image.isHidden = true
        return image
    }()
    
    private lazy var videoView: UIView = {
        let view = UIView()
        view.isHidden = true
        return view
    }()
}

Now wherever we need an image that could either be a regular image or animated we will create an ImageOrMP4GifView

@IBOutlet weak var imageView: ImageOrMP4GifView?

We wanted the call site to look like any other web image we load. Like I already mentioned we use Nuke for web images and we created an extension on UIImageView to minimize code duplication.

extension UIImageView {
    public func loadImage(with url: URL?, placeholder: UIImage? = nil, completion: ImageTask.Completion? = nil) {
        // Our NUKE code is here
    }
let imageURL = URL(String: "https://imageurl.com")
let placeholderImage = UIImage(named: "placeholder")
imageView?.loadImage(with: imageURL, placeholder: placeholderImage)

Now we need our ImageOrMP4GifView to display an image from the given URL or play a video in the AVPlayer. Inside of ImageOrMP4GifView we have this code,

public func loadImage(with url: URL?, placeholder: UIImage?) {
    imageURL = url
    reset()
    image.isHidden = false
    if MP4.shared.shouldConvertToMP4(url) {
        image.loadImage(with: convertURLForJPEG(url), placeholder: placeholder)
        MP4.shared.getMP4(url)
            .then(on: .main) {
                self.image.isHidden = true
                self.videoView.isHidden = false
                self.playVideoAtURL($0)
            }
    } else {
        videoView.isHidden = true
        // load image like normal
        image.loadImage(with: url, placeholder: placeholder)
    }
}

private func playVideoAtURL(_ url: URL) {
    // Init video
    videoPlayer = AVPlayer(url: url)
    videoPlayer?.isMuted = true
    videoPlayer?.actionAtItemEnd = .none

    // Add player layer
    videoPlayerLayer = AVPlayerLayer(player: videoPlayer)
    videoPlayerLayer?.videoGravity = AVLayerVideoGravity.resizeAspectFill
    videoPlayerLayer?.frame = videoView.frame

    // Add video layer
    if let videoPlayerLayer = videoPlayerLayer {
        videoView.layer.addSublayer(videoPlayerLayer)
    }

    // Play video
    videoPlayer?.play()

    // Observe end
    NotificationCenter.default.addObserver(self, selector: #selector(playerItemDidReachEnd), name: NSNotification.Name.AVPlayerItemDidPlayToEndTime, object: videoPlayer?.currentItem)
}
    
@objc private func playerItemDidReachEnd(notification: NSNotification) {
    self.videoPlayer?.seek(to: CMTime.zero)
    self.videoPlayer?.play()
}
    
public func reset() {
    image.image = nil
    videoPlayerLayer?.removeFromSuperlayer()
    videoPlayerLayer = nil
    videoPlayer = nil
    image.isHidden = true
    videoView.isHidden = true
}
    
public func cancelLoad() {
    MP4.shared.cancelMP4Load(url: imageURL)
}

loadImage(with url: URL?, placeholder: UIImage?) This method decides if an image or video should be displayed. Our MP4 singleton handles the decision of showing an image or video, and in this function we hide either the image or videoView view. playVideoAtURL(_ url: URL) is in charge of setting up the AVPlayer and playerItemDidReachEnd(notification: NSNotification) is needed to loop the video.

Our MP4 class needs the following imports to work,

import Bluebird
import Nuke

Bluebird handles our futures and promises and Nuke is being used for caching.

MP4 has a few opportunities for errors,

enum MP4Error: Error {
    case badURL
    case malFormedResponse
    case badData
    case failedRequest
    case failedToCreateCache
    case failedToCreateSymbolicLink
    case unknown
 }

Our MP4 class is a singleton and it keeps track of all in progress requests (in case any need to be canceled) and it has a data cache so that data doesn't get downloaded more than once.

public class MP4 {
    private var inProgressRequests = [URL : Promise<URL>]()
    private var dataCache = try? DataCache(name: "com.barstoolsports.ourUniqueName")

    public static let shared = MP4()
    private init(){}

We have a method to decide if the given URL is an animated image or not. For our urls, the animated images always end in .gif. This method may look different for your API.

func shouldConvertToMP4(_ url: URL?) -> Bool {
    guard let url = url?.absoluteString else { return false }
    return url.contains(".gif")
}

We can cancel requests,

func cancelMP4Load(url: URL?) {
    guard let url = url,
        let link = try? getUrlForMP4(url),
        let promise = inProgressRequests[link] else { return }
        promise.cancel()
        inProgressRequests.removeValue(forKey: link)
        dataCache?.removeDataAndSymbolicLink(for: link.path)
     }

And we can clear the cache,

public func clearCache() {
    dataCache?.removeAll()
}

A method to find if a file already exists,

private func fileExists(for url: URL) -> Bool {
    FileManager.default.fileExists(atPath: url.path)
 }

The following 2 methods you will see referenced in this code, but it has some Barstool API specific code that isn't important for this to all work, so I won't bore you with the details.

private func urlWithMP4Extension(_ url: URL?) -> URL? {
    // some fancy barstool code ๐Ÿ˜„
    return url
 }

private func getUrlForMP4(_ url: URL) throws -> URL {
    // some fancy barstool code ๐Ÿ˜„
    return url    
}

And finally, the method that pulls it all together,

func getMP4(_ url: URL?) -> Promise<URL> {
    guard var url = url else { return Promise(reject: MP4Error.badURL) }
    guard let dataCache = dataCache else { return Promise(reject: MP4Error.failedToCreateCache) }

    if let mp4URL = dataCache.url(for: url.path),
        let link = urlWithMP4Extension(mp4URL),
        fileExists(for: link) {
            return Promise(resolve: link)
        }
        do {
            url = try getUrlForMP4(url)
            if let inProgressRequest = inProgressRequests[url] {
                return inProgressRequest
            }
            let request = Promise<URL> { resolve, reject in
                URLSession.shared.dataTask(with: url) { data, response, error in
                defer { self.inProgressRequests.removeValue(forKey: url) }
                guard let httpResponse = response as? HTTPURLResponse else {
                    reject(MP4Error.malFormedResponse)
                    return
                 }
                 guard let data = data else {
                    reject(MP4Error.badData)
                    return
                 }

                 switch httpResponse.statusCode {
                     case 200...299:
                         if let promise = self.inProgressRequests[url], !promise.isCanceled {
                         dataCache.storeData(data: data, for: url.path)
                             .then { resolve($0) }
                         }
                     default:
                         reject(MP4Error.failedRequest)
                     }

                 }.resume()
             }
             inProgressRequests[url] = request
             return request

         } catch (let error as MP4Error) {
             return Promise(reject: error)
         } catch {
             return Promise(reject: MP4Error.unknown)
         }
     }

Using this approach we were able to remove a third party dependency. We were animating gifs using the great FLAnimatedImage framework.

Think my code sucks? Or, you want to talk more about this topic? Hit me up on Twitter, @TomRads