Introduction

First let me say, I am not a software developer, nor am I hardware developer. If I were, I may not be writing this post. I am merely a hobbyist and I would be absolutely overjoyed if a well-seasoned veteran in either of those areas read this post and had a solution for me. With that out of the way, a disclaimer: this is going to be a very long, rambling, post. I have spent the better part of the last year with my mind subtlety occupied by this subject. I think about it for at least a few minutes every day. This post is going to be a sort of dump of my collective learning on the subject, mostly just as an attempt at getting it out of my brain, and secondarily as a hope that perhaps someone will stumble across it and offer me a solution I may not have considered. There is no TL;DR, so either read it, or don’t.

For this not familiar with what an Ambilight Clone is, here is mine in action:

Background

About three years ago I, like many others, decided to build an Ambilight clone using a Raspberry Pi. I was quite successful. I can’t help but to feel I was on the early side of this trend but I hate sounding like a hipster, because I certainly didn’t come up with the idea or anything. I mean that only to say that I built my setup 100% from scratch, even to the point of having to heavily modify the original Boblight code to run on Arch Linux ARM with newer versions of FFMPEG, as well as be fast enough to cope with the larger than average number of LED’s I was using. While I won’t go into the deep details of this setup (I may do a retroactive write-up at some point), at a high level it required XBMC / Kodi via the Boblight plugin to feed a daemon on the Raspberry Pi over the network. This meant that my ambilight only worked with content that I could view on my HTPC, specifically inside Kodi. Technically I could have had any graphic on the OS (including the desktop or browser-based video streaming) feed the light, but this wasn’t exactly couch (wife) friendly, so I artificially limited myself to Kodi.

Somewhat recently I decided to switch from Kodi to Plex. There are many reasons for this but they are mostly irrelevant to the conversation. What is relevant is that Plex, out of the box, does not support plugins like Kodi does, specifically the Boblight plugin. To deal with this, I turned my old HTPC into a Plex Media Server, and upgraded my original boblight Raspberry Pi to a Raspberry Pi 3 which would run both the Plex Player, and a local instance of Hyperion. Instead of using my usual Arch Linux build, I went with RasPlex because it has Plex Home Theater and Hyperion support out of the box and I was able to get my existing setup back up and running on Plex very quickly and easily. Making this change forced me to revisit my long running ambilight setup. My setup was so reliable that I had not looked into it or kept on top of the technology since the day I first got it working. There was no need to follow the tech, because it worked.

As it turns out getting my light setup working with Plex was not the end, but only the beginning of my journey. I had just re-opened Pandoras box of my OCD mind. During the move over to Plex/Hyperion I realized a couple of things: One, I am very attached to my Ambilight, to the point where the idea of just getting rid of it never entered my mind. Two, I was tired of being limited as to the content which could drive the lights. While it is mostly fine that only the content I download works, as time moves forward I fear that the amount of content that falls into this category will be less and less. I find myself using services like Netflix, Amazon, and On Demand more and more all the time. If the industry continues advancing and provides me with reasonable, cost-effective ways to get content, I am happy to pay for it. If Netflix and Amazon gets movies the day they are released on Blu-ray available to stream without an additional rental fee, I would never download another movie. Three, in figuring out how to re-implement my light setup with Plex, I discovered a whole new population of people who had implemented an ambilight, as well as a lot of new software implementations I could play with.

Status Quo

One of the drivers of moving from Kodi to Plex was that I used Kodi on many devices, and the number of devices that Kodi supports out of the box is getting smaller. The list for Plex, however, is getting larger. Also, the client/server Kodi implementation is a bit of a hack and only works on your local LAN. The combination of the elegant client/server model for Plex and its vast array of native device support, lead me to test it out (among other options such as Emby), I liked it enough to switch. A perfect example of a place where Plex runs and Kodi does not (and likely will never, short of rooting the device) is on Smart TV’s. Despite being in the camp of people who do not much like Smart TV’s, as I would rather my multimedia device be decoupled from my display, for a variety of reasons, they are here to stay. Most TV’s today are smart, like it or not, and currently, Smart TV apps such as Netflix and Amazon Prime video are among the few ways one can actually consume 4k content. My new TV is a smart tv and happens to have the Plex app, which works wonderfully. I fully anticipate not using the smart tv functions in the future as the hardware will become outdated before the display does, but my point on the portability of Plex remains the same.

So anyhow, running Plex natively on my smart tv got me to thinking: “What if my Smart TV could drive my LED setup directly?”, or at the very least, “Is there a way I can run my ambilight system from other devices like Smart TV’s or Roku’s?” As much as I am a nerd and I love my embedded Linux boxes, Roku’s and other similar devices work quite wonderfully and you don’t have to implement hacks to stream Netflix or watch Blu-rays.

So the basis of the idea is simple: take the original Philips Ambilight concept, and remove it from the TV hardware itself. Meaning, make a kit that anyone could put onto any TV, and have it be able to function for any content that TV could display. This content could be OTA broadcasts, Smart TV apps, or video coming in via HDMI, etc. Since the current and next generation of Philips HDTV’s have an ambilight system built in, the whole thing is already patented, which is probably why it is left to the hobbyist community to tinker with as opposed to seeing another manufacturer implementation. One point to hold onto for later is that the Philips TV’s use Android TV to drive their current Ambilight technology.

Current Solution Set

Today there are several ways people implement ambilight clones. They all work, but all have the same limitations.

The first way is via ready-made kits such as the Light Berry and Lightpack. I love that people took the initiative to create these kits, however, they are simply glorified pre-packaged implementations of what I already built, with the same limitations. I will be the first in line when a kit comes out that can actually do what I want, but that is not the case with these. These kits are simply the same as my implementation but geared toward people without the time or inclination to build it themselves.

The second way is by doing something exactly like, or at least similar to, what I did. That is, use an embedded computer like a Raspberry Pi and drive the LED’s over SPI. The major differences here are in how the picture data is processed. This can be either via the network and a plugin, such as the Kodi Hyperion/Boblight plugin, or directly on the device via the Linux framebuffer in software (as I am doing today). This is just a homebrew version of the kits I just mentioned and is effectively exactly the same thing.

The third implementation is via a video capture device in-line with HDMI. This method is the absolute closest to what I want to achieve but is not something I am willing to invest time and effort into. Essentially it works like this. You get a cheap HDMI splitter capable of HDMI 1.4, 1080p video which happens to also completely ignore HDCP. As it turns out, almost every cheap HDMI splitter on Amazon simply ignore HDCP entirely, but they don’t advertise that fact for obvious reasons. On one side of the splitter is your TV, on the other is an HDMI-to-composite scaler that feeds a cheap USB video capture dongle connected, again, to a Raspberry Pi like device. I have a feeling there is noticeable latency involved in this setup, but that is not why I haven’t implemented it. Two years ago, I would have. The only reason I have not implemented it like this is because I want my system to be capable of the latest 4k UHD premium specs such as HDR, 10-bit color, etc. This means that I need to wait until a splitter / scaler that can ignore HDCP 2.2 as well as down sample a 4k60p signal quickly enough for low latency processing exists, or at least isn’t over $500. I am not sure how far on the horizon this might be, especially since HDFury was recently sued and told to stop selling all of their products. While I might not agree with the idea of HDCP, I would like to not have my system be non-compliant leaving me unable to watch Netflix.

The fourth, and most elegant solution I have seen yet, is an in-line HDMI FPGA LED controller. In my mind this is what an off the shelf commercial implementation of this technology would look like, with some additional software features built in. I have only seen one implementation of this, and it is far too complicated for the average joe to implement. I would try, but it is again limited to a non HDCP compliant 1080p signal, so unless I could implement a similar device which was HDCP 2.2 / HDMI 2.0 / 4k60p compliant, it wouldn’t suit my needs. I went on a several week long research binge after I discovered this and determined that it would be completely feasible to build this with my specs in mind, I simply don’t have the skills to do it.

I even went so far as to consider trying to buy a Sil9777 chip, development board, or a Samsung SEK3500U evolution kit just to try. I just don’t think I can see myself investing that sort of time/money/effort into it, but perhaps someone else will.

The fifth, and most novel-yet-stupid implementation I have seen is to use a camera. Some folks have used a webcam pointed at their TV screen to achieve the effect. In fact Philips own Hue system uses this method to control the Hue bulbs with TV content. The latency is massive and depending on what you want, this might be a great solution. For me, it is not. I like my ambilight to have rather high resolution and very low latency. If you are looking for a slower, low resolution ambient light, this would be great.

My Feature Wishlist

Required Features

• Does not require gaining unofficial root of a specific device. (Not that I have an issue with this, I just don’t want to be locked into a specific set of devices I can use. Nor do I want to risk bricking my $3000 TV)
• Extremely low-latency, accurate, ambilight effect from any video source.
• HDCP 2.2 / HDMI 2.0 Compliant such that any device (such as an NVidia Shield TV or a Roku 4) can drive the LED’s without breaking HDCP. Able to function around sources which implement UHD Premium specs such as HDR, 10-bit color, Wide gamut, etc.
• Reliable and somewhat elegant solution, meaning, not using 8 different converters/splitters/strippers/capture devices which all rely on some software behind it all.
• Ability to easily enable/disable the ambilight at any time. I don’t need the ambilight effect at 2pm when I’m watching a re-run of Roseanne.
• Implementation is relatively platform agnostic, meaning does not require a Samsung Smart TV or a Roku or any specific brand of device.
• As portable, upgradable, and future-proof as possible.

Optional “nice to have” features

• Software features such as app control of the lights for non-video related content. E.g. using a smartphone app to set the lights to run in some pattern when the TV is not even on for ambience. Hyperion can do this today.
• Ability to respond to music instead of video, similar to a music visualizer. I have not seen a clean implementation of this anywhere.
• Ability to respond to content that never leaves the TV, e.g. video from a Smart TV app that does not traverse the HDMI connection. Impossible today outside of Philips TV’s.
• Could be packaged and sold to your grandmother at best-buy, this will motivate someone to build it and make a lot of money. This would also mean it would not be a convoluted cluster-fuck of an implementation.
• Ability to also control additional lights, such as the Philips Hue, to control the ambient color of the entire room. This has also been done, and Hyperion can do it.

What I Have Learned

In my research I have learned a lot, about a lot of things, which I will do my best to summarize here. Not that anyone will read this, but if someone does, perhaps I can save you the time of digging too far into it yourself, since I already have. Most of what I learned is that there is a ton of shit I didn’t know existed with respect to embedded systems, display technology, signal processing, etc., etc. It has been very eye-opening.

1. Samsung Smart TV’s, as of this year, run the Tizen OS. LG TV’s run WebOS. These are embedded Linux. As such they 100% have the capability to fulfill my entire wishlist, technically. The Tizen Direct Rendering Manager spec illustrates this exact point. Beyond that, the Tizen developer app SDK also exposes the fact that the functionality is there, but developers are artificially limited any prevented from doing it. The Samsung Smart TV SDK “app lifecycle” prevents app developers from having apps run in the background. It is completely possible, but unless Samsung opens the API up more, or people root Tizen on Samsung Smart TV’s, a smart tv app implementation of Hyperion will not happen. I have even inquired with Samsung directly.
2. Many STB’s, Smart TV’s, and entertainment boxes (such as Tivo, NVidia Shield, Roku, Amazon Fire TV, etc.) run either embedded Linux or a custom Android build. Both of these platforms are 100% capable of running the requisite software to drive these LED setups. Again, the block is artificial and rooting these devices would be required. While I would be willing to buy an NVidia shield and root it, as far as I am aware that is not possible today, and it would be a very device specific implementation. Native, non-root android apps already exist, but are proprietary. Philips clearly developed this system for their Android based ambilight TV’s, and the Prism app that LightPack made for the OUYA are perfect examples.
3. A hardware implementation, like the one I talked about above, would be the most ideal. I guess the primary issue here is that if one were able to extract enough video data from the HDMI signal to do it, then they would also theoretically be able to pirate the video stream. Given that the point of HDCP is to prevent piracy and implement DRM, I am not hopeful that the HDCP spec will ever take into consideration an idea that one might want to simply extract the edge color data from the video stream for these purposes. As such I imagine that any hardware based implementation will have to bypass or break HDCP unless a large enough company wanted to develop this product and could actually license HDCP on the device.

Conclusion

There is a pretty massive demand for this type of setup, but not enough that the Philips Ambilight enabled TV’s are flying off the shelves. In fact, the entire idea is not very well known by the masses but everyone who sees it in my house wants to build one. It is my belief that whomever implements my entire wishlist will be a millionaire, and very likely get sued by Philips. I wish that person could be me, but I simply lack the abilities, time, and resources. As a result, I hope someone out there smarter than me pulls it off, so I can buy one. In fact, Philips should just decouple the technology from their TV’s and sell it, there is absolutely no reason they can’t do this today. Philips TV’s are not the most popular and they could make a lot of money just embracing the idea that Samsung and LG make better TV’s, but they make pretty cool smart lights.

I am extremely frustrated by my lack of ability to come up with a reasonable solution to this problem, especially since I know it is 100% do-able but I can’t implement it due to device specific artificial restrictions or DRM based limitations. Having a solution just outside of my reach has definitely been absolutely maddening. All of the devices we use to consume content, right down to the cable boxes, are capable, we just don’t have access.