Grayscale for the Greater Good
Ever since I worked on the greatest web graphics program ever, that’d be Fireworks for the uninitiated, I’ve been intrigued by Ted Turner’s idea of colorizing old movies, and image filters.
It wasn’t until the Fireworks 8 cycle that I really got a chance to start playing with image filters. I wrote several, but because I was pulled off Fireworks just a few months into Fireworks 8, only a couple of them actually made it into the final product. That didn’t end my interest in filters, but it did end my chance of publishing them.
I’ve also come to the conclusion that Ted Turner got it backwards. Instead of colorizing old movies, he should have been making all the new movies grayscale. As any artist can tell you, making images and/or movies black and white automatically makes them artsy. At least, that’s what all the artists I just made up in my head say.
The epiphany came today when I realized I should combine these two ideas, and create a filter that removes color. Well, that and show how to write Core Image filters. That might be interesting too.
Creating a project
I want to create a standalone Image Unit Plug-in. Fortunately, Xcode has a template just for that. To get this artistic revolution started, I select File > New Project… from Xcode’s menu. From here, I pick the “Image Unit Plug-in for Objective-C” template.
Next, I name my project “Grayscale,” because that’s what the plugin will do, and because I’m not a crazy Brit who insists on spelling gray with an “e.” The way they prattle on about it, you’d think they’d invented the language or something.
Modifying the description
Now that I have a project, the first thing I want to do is modify the description property list. The Description.plist is what tells Core Image what the parameters for the filter are, the category of the filter, and where to find the kernel, which, surprisingly, isn’t always at KFC.
In order to find my filter later to use it, I need to give it a meaningful category. Apple has several predefined categories that I have to choose from, thus cramping my artistic style. Despite this, I can still improve on the default choice of CICategoryStylize, by changing it to CICategoryColorEffect.
After my change, the Description.plist category section looks like this:
<key>CIAttributeFilterCategories</key>
<array>
<string>CICategoryColorEffect</string>
<string>CICategoryVideo</string>
<string>CICategoryStillImage</string>
</array>
The most important thing to modify the in Description.plist is the description of the filter’s parameters. It turns out that I don’t need any input parameters for this colorless revolution, with the exception of the source image. The default Description.plist comes with three parameters, so I’ll delete the last two parameters.
That concludes everything I need to do with Description.plist. However, I still need to change the display name that shows up in the UI for my filter. For that I open up Description.strings:
"MyKernelFilter" = "Demo CIKernel Only Filter";
"inputScale" = "Scale";
"inputGreenWeight" = "Green Weight";
Since I don’t have any user inputs, I can ditch the two last strings, inputScale and inputGreenWeight. MyKernelFilter is the name of my filter, so I’ll change it to “Grayscale.”
"MyKernelFilter" = "Grayscale";
That does it for all the meta-information for my filter plug-in.
Kernel
Now that I’ve set up all the resources for my filter, I need to write the actual code. The kernel code is written in what is called Core Image Kernel Language, which is a subset of the OpenGL Shading Language.
To start, I open up MyKernelFilter.cikernel, and delete it’s current contents. In its place, I add the following code:
kernel vec4 grayscaleKernel(sampler image)
{
// Get source pixel
vec4 p = sample(image, samplerCoord(image));
// Calculate the intensity
float intensity = clamp(0.3* p.r + 0.59* p.g + 0.11* p.b, 0.0, 1.0);
// Set the destination pixel based on intensity
return vec4(intensity, intensity, intensity, p.a);
}
It doesn’t really matter what I name the function, as long as it is tagged with the kernel keyword. The kernel, in this case, acts as a map from the source pixel value, to the destination pixel value. You’ll note that it returns a vector of four elements (Red/Green/Blue/Alpha) and takes the source image as the only parameter. If I had other parameters, they would come after the image parameter, just like in the Description.plist file.
The first line simply retrieves the pixel from the source image:
vec4 p = sample(image, samplerCoord(image));
As established earlier, image is the source image passed in as a parameter. samplerCoord() returns the coordinates of the source pixel inside the source image. sample() does what it says, and will return a single pixel value given an image and coordinates.
The second line calculates the intensity of the source pixel:
float intensity = clamp(0.3* p.r + 0.59* p.g + 0.11* p.b, 0.0, 1.0);
To do this I add the weighted RGB components together. The reason behind the weights goes beyond what I want to cover here, but I use intensity calculation from the YIQ color model. For safety reasons I clamp the value to between 0 and 1, because each color component has to stay in that range.
Finally, I return the value of the destination pixel:
return vec4(intensity, intensity, intensity, p.a);
I’m simply setting each of the RGB components to the intensity, which will make the color a shade of gray. Also note that I’m leaving the alpha channel alone.
That’s it as far as the coding goes. I just need to select Build > Build from the menu to complete my filter.
Making cone cells obsolete
Before I can do anything with my Grayscale filter, I need to install it. Core Image looks in a couple of different places, but the easiest folder to stick it in is: ~/Library/Graphics/Image Units/.
Although I could use Quartz Composer to try out my filter, I like the Core Image Fun House, because it’s simpler and more to the point. It can be found at /Developer/Applications/Graphics Tools/. When I open the Fun House, I’m prompted to pick a source image. I usually pick the Wolf.jpg, whom I’ve nicknamed “Rentzsch”, for reasons that escape me.
After picking an image, I click the plus button, and pick my Grayscale filter from under the Color Effect category.
And there you have it. A very artistic, extraordinary, gray wolf, of whom there are none like. Hmmm… maybe I should have created a green filter.
Conclusion
Although I avoided Quartz Composer for this simple example, it is pretty much required for any filter of sufficient complexity. When building, Xcode doesn’t actually compile the kernel or do any type of checking. As far as I can tell, only the Core Image Kernel tool in Quartz Composer will do checking. Unfortunately, Quartz Composer doesn’t allow me to inspect variables or step through statements. It only does basic syntax and type checking.
Also, the shading language, especially with Apple’s restrictions, is pretty limiting. Even with simple kernels, I struggled to get them to work with the kernel language.
Despite the restrictions, I’m having a lot of fun playing around with Core Image kernels. I’ve written a few others that I may, at some point in the future, show off here. Until then, have fun viewing the world through the eyes of a dog.