The computer generated image has fascinated me since I was kid. I distinctly remember as an 8-year old boy the amazement of novelty when my mom borrowed a Commodore 64 and a colour monitor back in the 1980es for a weekend. It came with the educational programming language Logo on a floppy disk, which allows you to create abstract drawings by means of controlling an imaginary turtle. Surely the graphics were extremely primitive back then, but when this was your first introduction to the world of computers it seemed like pure magic and love at first sight.
Photo credit: Chris Lawton.
There was something about using the human creative power in making computer algorithms for generating or manipulating pixels that immediately resonated with me. Eventually I got my own computers, and growing up on the quiet Danish countryside meant few distractions and plenty of time to explore this passion. In retrospect, it clearly influenced my friendships, education and working career in amazing ways. Once a pixel junkie, always a pixel junkie, as the old saying goes…
Why we created Pixop
Fast-forward to the Spring of 2018. Morten and I founded Pixop the year before with the intention of eventually making it our full-time gig. We had worked closely together in the past and were drawn to the possibilities and recent advances in the field of machine learning and AI for improving the image quality of still images and video.
The idea of building a cloud platform was literally conceived over a steaming hot cup of espresso at Morten’s house. We were old-school brainstorming sensible proposals for how to put our technology to good use. The idea that really stuck with us was upscaling and enhancing video such as old movie classics. Loving great stories ourselves and enjoying the benefits modern technology gives us in terms of high-definition displays, it seemed to us that creating something like this would be a valuable service to the world and Google Search indicated to us that nothing similar out there already existed. So it was settled! We decided to venture into making algorithms and services that produce the most appealing digital imagery possible.
Today we are helping a range of video professionals. By giving people access to ground-breaking technology, we are adding technical value to existing productions thereby freeing resources that allow the creation of new stories. We all want more stories to be told by the best creative minds out there. A win-win for both consumers and producers alike.
Photo credit: Andre Hunter.
Great stories deserve the best presentation
In no particular order here are the top reasons why we doing this:
- Like most people, we enjoy watching movies and TV series in the best possible technical quality
- Today there are often no inexpensive solutions to “fixing” degraded digital video productions
- Many great stories were produced in a resolution lower (typically SD) than what is currently the state-of-the-art for displays
- The gap between the technical quality of older productions and the capabilities of current display technology is increasing every day
- Recent research shows that machine learning processing results can be made visually more impressive than what’s achievable for a human using video editing software
- Making machine learning easily accessible is hard, require innovation and making smart compromises
- We genuinely love creating software services — it’s what we have been doing for our entire professional career
- Post-production requires manual labor and in many cases, it won’t get done today because of the cost involved
- There is an increasing demand for quality content on streaming services
In summary, we want to serve the world by improving the digital video viewing experience in an affordable and easily accessible way using machine learning.
Powerful tools for video aficionados
Samantha, head of streaming services, is happy about getting the network’s dated content back on air in improved quality and higher resolution
We wanted to make something for a specific target group, being aware that products which appeal to the widest possible audience generally fail due to lack of focus. A small core who loves you is better than a bunch of people who thinks you are merely “ok” as Seth Godin puts it. In other words, it was never our intention to create a mass consumer product. Instead a range of self-taught amateurs to hardcore industry pros of 35 years should benefit from Pixop Platform. Here are some specific types of users and example use cases we had in mind:
- Video hobbyists occasionally upscaling drone footage, their latest home cooking adventures or nature videos to a higher resolution
- Video production companies upgrading their backlog of movies that lack technical quality compared to today's standards and who want to keep potential buyers in the loop
- Post producers working on an HD documentary and that include clips of SD archive footage shot in the 1990s
- Broadcasting corporations airing a TV series next week in prime time that was produced 20 years ago
- Streaming services making a catalogue of classic cartoons available on their OTT channel
Having prior technical knowledge of digital video processing and formats is necessary to make the most of the platform. We believe this is being reasonable considering the target audience and since we apply augmented intelligence in several areas to aid our users in making sensible choices.
Pixop: Dashboard view
OK, tell me more…
In a nutshell, Pixop Platform is a browser web app offering an intuitive platform for storing, transcoding and machine learning powered processing video in the cloud. All that is required is a modern internet browser with solid support for JavaScript, HTML5 and CSS3.
The platform includes project based video asset management, video quality analysis and a range of collaboration options to engage with your team and clients.
Watching the demo video at the top of this post is probably the best way to get an understanding of the product.
The typical workflow consist of five simple steps:
- Upload and ingest source video
- Confirm ingested video quality
- Process video
- Evaluate result directly in your browser
- Download processed video and/or share comparison link with a client
No investment in hardware required since all the magic happens in the cloud. All you need is a web browser and an internet connection. No software installation or set-up needed.
Pixop: Video page
Going behind the scenes
Our entire solution is currently entirely hosted at AWS Ireland — including some parts on Heroku. The decision was a no-brainer for us really. The geographical location is a sweet spot knowing most of our customers are either in Europe and the US. This is also the European datacenter that generally receives the most love in terms of getting early access to new features and services. As a startup company we recognize the importance of agility, the value of third party services and speed to market.
The web app facing the end-user is React based and the power of JavaScript, HTML5 and CSS3 drive the look and feel. WebSockets are used to keep team members working concurrently on the same projects synchronized, so that everybody sees when new videos are being uploaded, processed, etc.
A number of different backend AWS services are being leveraged to ensure scalability, reliability and a seamless experience. Video processing is demanding and a perfect fit for cloud deployment due to the massive amount of computations and storage required. We rely on essential managed core services such as S3 for object storage, Lambda and Batch for “serverless” computing, CloudFront for content delivery, DynamoDB as our key-value NoSQL database and EFS as workflow storage during ingestion and processing. The obvious benefit is that it allows us to focus on creating better algorithms for enhancing your imagery, while leaving the burden of creating the best possible infrastructure to the experts at AWS.
A fundamental concern for us is your data security and privacy. We believe video masters should be regarded as crown jewels. Besides the basics like GDPR compliance, using OWASP secure coding practices during development and storing passwords in a salted-hashed form, we employ encryption at rest with all data stored on S3. We further ensure all data traffic sent over the wire is strongly encrypted — even internally at AWS. As a final example, cryptographic constructions such as URL signing is used to ensure time limited access when sharing a comparison link.
Pixop: Processing dialog
Supported video formats
A frequently asked question is “What formats do you support?” It’s a completely valid question that is not answered on our website today, although it will be soon.
In general we can ingest anything FFmpeg 4.2 supports, i.e. all well-known codecs. Refer to FFmpeg’s documentation for the full list, but in general it is a safe bet that we are able to decode and process your video.
Before completely uploading a file to the cloud, we test the format to ensure the container is supported by analyzing the first kilobyte. We support MPEG-4, AVI, MXF, Matroska, QuickTime, MPEG and even some less widely used formats like ASF and Ogg.
Target formats for processing is limited to a selection of the most popular codecs and containers. We decided to keep options limited in order to make sure what we are offering actually works. Getting the combination of container, encoder, target resolution, target bitrate, chroma subsampling and pixel format working correctly is not trivial. Due to the specifications of codecs like Apple ProRes and DNxHD/HR only certain settings are actually valid, but we made an effort to promote ease of use by removing some complexity around this in how the UI is designed. Also, the target bitrate is inferred from bit rate of original video and target resolution.
Without further ado, these are the output options we currently support…
- Containers: QuickTime, Material Exchange Format (MXF), MPEG-4, MPEG-TS
- Codecs: H.264 / AVC, Apple ProRes, Avid DNxHD/HR, MPEG-2 video, H.265 / HEVC
- Pixel formats: All commonly YUV chroma subsampling types and bit depths are supported
We are happy to take requests if there is something you need which is not covered by this list.
Note that we don’t do audio transcoding — audio streams, subtitles and metadata from the source video are multiplexed into the target video “as is”.
Statistics page on Pixop
No free lunches
We believe in a fair, transparent business model. AWS infrastructure costs associated with storing your data in the cloud and downloading to the internet are invoiced on a monthly basis along with a basic subscription fee and any processing performed. The projected monthly costs can be monitored using the stats page and we offer a quality guarantee so you can process in an affordable way while keeping track of your expenses.
For now processing jobs fall into one of two categories in terms of pricing: vanilla video processing ($0.012/GP) and Pixop Super Resolution upscaling ($0.24/GP). Processing jobs can include any number of pre-processing filters such as de-interlacing, de-noising and bicubic upscaling, as well as transcoding to a different format.
For AWS Ireland’s current pricing checkout Amazon’s website for S3 (data storage) and CloudFront (download) — as of December 2019 the prices are $0.023/GB and $0.09/GB respectively. We do not intent to make any financial gains on infrastructure services and therefore charge our end users the same as AWS’ going rates. In other words, the cost of data storage and internet download on Pixop Platform will be the same as running your own AWS account.
Monthly plans are $9.99 for a single user, $29.99 for a small team of up to five members or $99.99 for an enterprise plan with multiple teams and unlimited users.
It’s not a requirement to buy a plan to test out if Pixop Platform is useful to you. Watermarked previews limited to 60 seconds can be created for free and they will be auto-deleted after one day.
Pricing example
As a basic pricing example, let’s take an independent video producer:
- Single user
- 100 GB of active storage (average per day)
- 20 GB downloaded data
- 124.4 GP of Pixop Super Resolution upscaling (10 minutes UHD 4K @ 25 fps)
The monthly charge amounts to $43.95 using the rates mentioned above.
Peeking into the future
Photo credit: Drew Graham.
Although linear TV channels certainly still have their place, we all know that increasingly flexibility and options are going to be needed. OTT and streaming video define the future of TV and content is going to be king. We strongly believe there is a need for what we are offering and the consumers are ready for it.
We have several exciting new features and services in the development pipeline for next year and beyond.
Here are some highlights of what to expect:
- Additional algorithms like end-to-end video restoration, SDR to HDR conversion, flickering removal and automatic quality control
- APIs for image processing, video processing, thumbnail generation, etc. along with WS clients for the most popular programming languages
- XDCAM and IMX encoding support
- Image comparison as an alternative to in-browser video comparison
- File upload and download via FTP and SFTP for enterprise integrations
- Interlacing agnostic processing and encoding
- Processing presets
- Bulk processing on an entire project via just a few clicks
Let us know what else you need by dropping us a line on Intercom. We are always open to suggestions and improving.
A team on a mission
We love our journey and it is empowering to feel our vision become real. We are learning new every day by failure and success — with our vision of helping video consumers and producers we are used to overcome obstacles. Those who have engaging stories to share deserve powerful tools that create the best possible video viewing experience for us all. We are committed to provide just that.
Experience it yourself
Come and sign up on Pixop Platform for free and test our algorithms on your own footage. Go to: pixop.com.
Big thanks to Innovation Fund Denmark, NVIDIA Inception program and everyone else involved in making Pixop a reality.