Unfortunately, I’ve not had much luck with Quartz Composer in way of meaningful compiler errors. Sometimes it returns good errors, sometimes inscrutable ones.

Composer: Wait, wait, I remember this… it’s COBOL, right?

However, I’ve found debugging OpenCL kernels to be much easier thanks to the invention of a function I like to call “printf.” If you run your kernel on the CPU, then you can print out values just like in C. With debugging solved, I just needed to figure out how to find compiler errors without having to run my code.

Unlike Core Image, OpenCL actually requires you to compile your kernel code explicitly before using it. The API’s required to do this will return nice compiler error messages and everything. The obvious thing to do would be wrap up these API’s in a command line tool and invoke the tool from Xcode, which is what I did.

The Code

You can download the Xcode project for clc, the OpenCL Compiler here. It requires Mac OS X 10.6 Snow Leopard and the associated Xcode tools. To install, just build the project and copy the resulting binary into your path somewhere. I used ~/bin.

Most of the code is self explanatory, with the possible exception of the output. Unlike most compilers, clc doesn’t output object code because the host and runtime CPU/GPU’s could easily be different. It could simply copy the source code as is for the output, but that was a little too obvious for my tastes. Instead clc compresses the source code and stuffs it into a binary plist. It has the slight benefit of potentially being smaller on disk, and making your source code negligibly less accessible to prying eyes, if you’re into that sort of thing.

There are definite possibilities for improvement. For example, there could be an option to encrypt the source code to be more “secure.” Or it could store the binary code generated by the host machine in the output on the chance that the host and runtime machines have the same hardware.

Setting up Xcode

To use clc, you’ll need to set up a build rule in your app’s target settings to run all OpenCL source files through it. Selecting the menu item Project > Edit Active Target, and then selecting the Rules tab, should land you here:

Add another rule, set it to process OpenCL source files using a Custom script. For the custom script enter:

~/bin/clc ${INPUT_FILE_PATH} -o ${TARGET_BUILD_DIR}/${PRODUCT_NAME}.app/Contents/Resources/${INPUT_FILE_BASE}.clar

Finally, you’ll need to tell Xcode where you’re putting the output file, which is:

${TARGET_BUILD_DIR}/${PRODUCT_NAME}.app/Contents/Resources/${INPUT_FILE_BASE}.clar

By default, Xcode won’t put OpenCL source files (.cl) into the target, meaning they won’t get compiled. For each of your OpenCL source files
you’ll need to explicitly add them to your target. There are several ways to do this, but the easiest is probably selecting all of them, and doing a Get Info (File > Get Info). Switch to the Targets tab and check the target they should be compiled into.

Runtime Code

At runtime, it’s pretty easy to retrieve the original OpenCL source code using the + (NSString *) openCLSourceWithData:(NSData *)data error:(NSError **)error; method on ONOpenCLArchive. However, there are a couple of convenience methods on NSBundle that makes retrieving the kernel easier. For example:

#import "ONOpenCLArchive.h"

NSString *kernelSource = [[NSBundle mainBundle] openCLResource:@"MyKernel"];

That’s pretty much all there is to it.

Conclusion

Although not ground breaking, this little tool has certainly been helpful to me by finding compiler errors at compile time instead of runtime. Hopefully it will be useful to other people as well.

This year they aren’t content with bringing conference justice to only the UK, so they’re branching out to the good ‘ole U.S. of A. by way of Atlanta, GA. To help you decide which conference you should attend, US or Europe, I’ve prepared the following quiz:

1. What is your opinion of Seattle?

   1. It’s nice, but isn’t nearly rainy or dreary enough.
   2. I like their coffee.
   3. The residents have too many teeth.

2. Describe your driving habits

   1. I like to drive on the left side of the road.
   2. I like to drive on both sides of the road.
   3. I like to merge right six lanes without signaling while going 147 mph on the off ramp and giving the finger with both hands in my black Camaro.

3. The pinnacle of human achievement is…

   1. Afternoon tea
   2. Sliced bread
   3. Hee Haw

Scoring: Give yourself -1 points for any 1 answer, 0 points for any 2 answer, and 1 point for any 3 answer.

If you scored is less than zero, you should attend NSConference Europe; if greater than zero, NSConference USA. If you scored exactly zero, you are truly a cultured individual and should attend both.

Personally, I’ll be attending both, and not just because of peer pressure and insightful quizzes. I’ll be presenting a talk on how to implement a watercolor brush using Core Image and OpenCL and maybe some duct tape. If you enjoy the graphics articles that I post here, you’ll probably enjoy my presentation. If not, I hear Steve “I’m Batman” Scott does a mean Adam West impression.

However, one of my frustrations with working with other app’s AppleScript interfaces was trying to figure out how the interface was intended to be used. Sure, the AppleScript Editor would show me all the actions and classes, but it isn’t always obvious how things are supposed to fit together. Something that would help in these cases is AppleScript recording. I could record the app performing the actions I cared about, then examine how the app itself used the actions and classes. Unfortunately, it seems like only the Finder and BBEdit ever got around to implementing AppleScript recording.

In the hopes of increasing the number of apps with AppleScript recordability, I’m going to document my approach to implementing it. For brevity, I’ll assume you’ve already an AppleScript interface implemented for your Cocoa app.

Thinking Big Design Thoughts

If you have an AppleScript interface for your application, you may think of your app architecture as something like this:

Here your AppleScript interface and graphical interface are independent peers, and both modify your model classes directly to accomplish your application’s tasks. Each interface is separate and largely ignorant of the other.

However, when implementing AppleScript recordability it is helpful to think about your app’s architecture in a different way:

In this case the GUI is dependent on, and implemented in terms of, the AppleScript interface. The general guideline is that anything the GUI does that mutates or alters the model goes through the AppleScript interface. However, if the GUI simply needs to access or get information from the model, it would go directly to the model, not through the AppleScript interface. Accessing the model can happen at seemingly random times to the user, and spamming the AppleScript Editor with these accesses when recording only confuses the user.

Suppose an application has table view and a button that deletes the currently selected item in that table view. The table view data source would be implemented the standard way, simply going directly to the model, bypassing the AppleScript layer. However, the delete button, since it alters the model, would be implemented by invoking the AppleScript delete command on the object represented by the current table row.

This design has some other benefits besides recordability. Notably it helps test your AppleScript interface design and implementation. If you find that implementing a GUI feature in terms of your AppleScript interface is impossible or difficult, congratulations, you found a bug! Also, merely testing your GUI also exercises your AppleScript interface. It is not a replacement for testing your AppleScript interface explicitly, but it certainly helps.

Implementation Hardships

Everything I’ve talked about so far hasn’t been all that novel, and probably has been met with large bucket fulls of “well, duh”s by anyone who’s ever looked into implementing AppleScript recording. The problem isn’t thinking about how to design in recordability, but actually implementing recordability.

As things stand now, doing something as simple as “invoking the AppleScript delete command on the object represented by the current table row” is incredibly involved and painful. You have to manually build up the AppleEvent that represents the delete command and the target model object using functions like AEBuildAppleEvent or classes like NSAppleEventDescriptor. Then you have to remember to target your app by specifying the kCurrentProcess process serial number (specifying kCurrentProcess as the ProcessSerialNumber is currently the only way to enable recording. Bundle identifiers, urls, and pid_t’s do not work.), and parse the AppleEvent you get back into something useful. You’d have to do this for every property or method on your model object you want exposed for recordability.

Dreaming of Ponies

The thing is, the Cocoa runtime has a lot of the AppleScript information from your SDEF file at its disposal and could theoretically generate these interfaces for you. In the ideal hypothetical situation, invoking the delete action via AppleScript inside your app could be as simple as:

// Suppose ONEmployee is our model object, with the appropriate AppleScript interface implemented
ONEmployee *employee = [_employees objectAtIndex:0];
ONEmployeeASProxy *proxy = employee.appleScriptInterface;
[proxy delete];

Here, any object that implemented the objectSpecifier method for AppleScript support would automatically get an appleScriptInterface property. The object returned by appleScriptInterface would be a proxy object implementing the same methods and properties as the original object. The proxy object would implement these methods by building up the appropriate AppleEvents, sending them, and parsing the resulting event back into a usable object.

Apple actually gets tantalizingly close to this with the Scripting Bridge. Outside users of your app can run your SDEF file through the sdp command line tool and get a nice Objective-C interface of proxy objects that build, send, and parse AppleEvents to and from your app. However, there is not currently a way tell these proxy objects to target kCurrentProcess, or to initialize one of the proxy objects by passing in a model object that implements objectSpecifier. (I’ve written this up as rdar://problem/7359646.)

Harsh Reality

Since I didn’t want to wait on Apple to extend the Scripting Bridge to make my life easier, I decided to write a couple of classes to help out. You can download the code here. The code is under the MIT license.

The classes work similar to the SBObject and SBElementArray classes in the Scripting Bridge framework. Using these classes, you can invoke the delete AppleScript method like so:

// Suppose ONEmployee is our model object, with the appropriate AppleScript interface implemented
ONEmployee *employee = [_employees objectAtIndex:0];
[ASObj(employee) invokeCommand:@"delete"];

The ASObj function creates an ASObject proxy object for any NSObject that implements objectSpecifier. invokeCommand takes care of marshalling the parameters into an AppleEvent, sending it, and unmarshalling the return value into an NSObject. The name of the command is the name of the name used in AppleScript, not the Cocoa implementation.

invokeCommand can take parameters, although it gets more tricky:

ONEmployee *employee = [_employees objectAtIndex:0];
[ASObj(employee) invokeCommand:@"giveRaise" with:[NSNumber numberWithInt:10], @"Percent", nil];

First, the parameters must be named (it’s not done by parameter order), and those names must match the Cocoa Key in the SDEF, not the user visible parameter name. Second, the marshalling code (from random NSObjects to AppleEvents) is a bit sparse. I’ve only added code for the types I needed for my project. If you use it, you may need to add support for other types. The same goes for return values; I only added support for the types that I use.

ASObject also has support for properties. For example this marks an employee as exempt:

ONEmployee *employee = [_employees objectAtIndex:0];
[ASObj(employee) setObject:[NSNumber numberWithBool:YES] forProperty:@"isExempt"];

The same type restrictions for parameters apply to properties as well. The Cocoa Key for the property must be used here, the same as parameters.

Elements also have basic support, which is where the ASElementArray comes in. Right now the only interesting thing to do with an element array is retrieve a reference to a specific element:

ONEmployee *employee = [_employees objectAtIndex:0];
ASObject *dependent = [[ASObj(employee) elementForKey:@"dependents"] objectAtIndex:0];
[dependent setObject:[NSNumber numberWithBool:YES] forProperty:@"insured"];

Unlike other methods, ASElementArray’s objectAtIndex does not execute an AppleEvents or otherwise take any actions. Instead it constructs an object specifier (i.e. an ASObject) for the given element.

The code is still somewhat rough and incomplete, but should help with anyone wanting to implement AppleScript recording. If nothing else, it should serve as a starting point or sample code for anyone rolling their own solution.

The sharpen tool has the following effect on this tweed pattern:

Notice the stripe through the middle where the edges are more strongly defined.

The blur tool has the opposite effect of sharpen on the tweed pattern:

Implementation Overview

Like the sponge tool, the sharpen and blur tools are subclasses of the FilterBrush class introduced in the dodge and burn tutorial. However, unlike sponge, they do require a slight modification to the GenericBrush class to work.

Since almost all of the code in this tutorial has been covered before, I’ll just highlight the single change to the GenericBrush class, and the new Sharpen and Blur classes. If you want to see everything, download the code.

Changing GenericBrush

If you recall, at the very beginning of this article I mentioned that sharpen and blur tools typically allow the user to modify the blend mode. This is the blend mode used when stamping the filtered image back on to the original image. GenericBrush contains the code that handles the stamping, so it will need to be modified.

We change the render:at: method for GenericBrush to be:

- (void) render:(Canvas *)canvas at:(NSPoint)point
{
	CGContextRef canvasContext = [canvas context];
	CGContextSaveGState(canvasContext);

	CGPoint bottomLeft = CGPointMake( point.x - CGImageGetWidth(mMask) * 0.5, point.y - CGImageGetHeight(mMask) * 0.5 );

	// Our brush has a shape and soft edges. These are replicated by using our
	//	brush tip as a mask to clip to. No matter what we render after this,
	//	it will be in the proper shape of our brush.
	CGRect brushBounds = CGRectMake(bottomLeft.x, bottomLeft.y, CGImageGetWidth(mMask), CGImageGetHeight(mMask));

	CGContextClipToMask(canvasContext, brushBounds, mMask);
	CGContextSetBlendMode(canvasContext, [self blendMode]);
	[self renderInCanvas:canvas bounds:brushBounds at:point];

	CGContextRestoreGState(canvasContext);
}

The only change made is the call to CGContextSetBlendMode that allows brush subclasses to determine the blend mode to use. The default implementation is simply:

- (CGBlendMode) blendMode
{
	return kCGBlendModeNormal;
}

That’s it for GenericBrush.

Sharpen Tool

The sharpen tool has just two parameters: mMode, which is the blend mode, and mStrength which determines how strong the sharpen effect is applied. They are initialized in init like so:

- (id) init
{
	self = [super init];
	if ( self != nil ) {
		// Set the default values for our parameters
		mMode = kCGBlendModeDarken;
		mStrength = 1.0;
	}
	return self;
}

Here are some examples of the sharpen tool parameters:

mMode	mStrength	Result
kCGBlendModeNormal	0.5
kCGBlendModeNormal	1.0
kCGBlendModeDarken	1.0
kCGBlendModeLighten	1.0

The blend modes typically offered for a sharpen tool are kCGBlendModeNormal, kCGBlendModeDarken, kCGBlendModeLighten, kCGBlendModeHue, kCGBlendModeSaturation, kCGBlendModeColor, and kCGBlendModeLuminosity. In the examples above, kCGBlendModeDarken causes the sharpening to only affect light pixels and kCGBlendModeLighten only affects dark pixels.

The heart of the sharpen tool is the filter it creates. The code for that is:

- (CIFilter *) filter
{
	// We need to create and configure our filter here
	CIFilter * filter = [CIFilter filterWithName:@"CIUnsharpMask"];
	[filter setDefaults];
	[filter setValue:[NSNumber numberWithFloat:mStrength * 100.0] forKey:@"inputRadius"];
	return filter;
}

You’ll notice that I’m using the unsharp mask filter to do the sharpening. I should point out that most image editors don’t use this filter to do the sharpening. Instead, they seem to use a convolution filter to do the sharpening. However, the convolution filter means it is easy for the user to overdue the effect. Unsharp mask does not have that problem, plus unsharp mask is provided by the system.

The final bit of the sharpen tool is returning the correct blend mode:

- (CGBlendMode) blendMode
{
	return mMode;
}

Blur Tool

The blur tool is very similar to the sharpen tool. It even takes the same parameters. The only real difference is the filter that it creates, so that’s what I’ll present first:

- (CIFilter *) filter
{
	// We need to create and configure our filter here.
	CIFilter * filter = [CIFilter filterWithName:@"CIBoxBlur"];
	[filter setDefaults];
	[filter setValue:[NSNumber numberWithFloat:mStrength * 7.0] forKey:@"inputRadius"];

	return filter;
}

Here I use the built in CIBoxBlur. mStrength ranges from 0 to 1, inclusive. I set the coefficient of 7 simply by trial and error until I got something that “looked right.”

Here are some examples of the blur tool:

mMode	mStrength	Result
kCGBlendModeNormal	0.5
kCGBlendModeNormal	1.0
kCGBlendModeDarken	1.0
kCGBlendModeLighten	1.0

Conclusion

Sharpen and blur are the last of the common filter brushes. Enjoy, and download the sample code.

Overview

In concept, the sponge tool is a simple tool. It either increases or decreases the saturation of an image. Saturation describes how intense a given color is. 0% saturation means the color is gray. In the implementation that I will show, the sponge tool will behave like a brush.

Thus, if I have a simple red square with about 50% saturation, the sponge tool in the saturation mode affects it like so:

The desaturation mode has the opposite effect on the same image:

In addition to the saturate/desaturate mode, the sponge tool has a “flow” parameter, which determines how much to saturate or desaturate the image.

Implementation Overview

The sponge tool is simply a subclass of the FilterBrush class, which I created for the dodge and burn tutorial. In fact the only difference between this tutorial and that one is the new subclass Sponge. For that reason I’ll only be covering the Sponge class in this tutorial. For an explanation as to how the rest of the code works, please refer back to the dodge and burn tutorial.

As always, I’ve provided sample code for this article.

Sponge

Parameters

The sponge tool has two parameters which are initialized in the init method:

- (id) init
{
	self = [super init];
	if ( self != nil ) {
		// Set the default values for our parameters
		mFlow = 0.5;
		mMode = kSpongeMode_Desaturate;
	}
	return self;
}

The two parameters for the sponge tool are mFlow and mMode.

• mFlow Flow determines how strong the saturation or desaturation effect is applied to the image. It ranges from 0.0 to 1.0, where 0.0 means the effect isn’t applied, and 1.0 is where the effect is at its strongest.

  Sponge in saturate mode, varying flow:

  mFlow	Result
  0.25
  0.5
  1.0

• mMode Mode determines if the sponge saturates or desaturates the image. It has only two settings: saturate or desaturate. Saturate increases the color’s saturation, while desaturate decreases saturation.

  Sponge examples:

  mMode	Result
  kSpongeMode_Saturate
  kSpongeMode_Desaturate

In the examples I used a red rectangle with about a 50% saturation. That way, both saturation and desaturation effects will show up on it.

Creating the filter

If you recall from the dodge and burn tutorial, the only real responsibility of a FilterBrush subclass is to create a filter that will be applied to each brush stamp.

- (CIFilter *) filter
{
	// We need to create and configure our filter here. CIColorControls has controls
	//	for saturation which is what we care about.
	CIFilter * filter = [CIFilter filterWithName:@"CIColorControls"];
	[filter setDefaults];
	if ( mMode == kSpongeMode_Saturate )
		[filter setValue:[NSNumber numberWithFloat:1 + mFlow] forKey:@"inputSaturation"];
	else
		[filter setValue:[NSNumber numberWithFloat:1 - mFlow] forKey:@"inputSaturation"];

	return filter;
}

Unlike the dodge and burn tools, I don’t use a custom filter. Instead, I use the system provided CIColorControls, which has a parameter for saturation. The saturation parameter has a range of 0 to 2, where 1 is the identity value. Thus, if I’m trying to saturate the image, I add 1.0 to mFlow, and if I’m trying to desaturate the image, I subtract mFlow from 1.0.

That’s it. The rest of the heavy lifting is done by the framework I created in the dodge and burn tutorial.

Conclusion

The main thing this tutorial demonstrates to me is that the time spent building a framework for filter brushes paid off. It should help in the future if I decide to tackle the blur and sharpen brushes as well.

Hopefully you found this post enlightening. Don’t forget to download the source code.

Although I’m not a lawyer and can’t offer any legal advice or support, I can do something. First, I can express my displeasure at someone who claims to be part of the Mac developer community treating Stepwise, or any other developer for that matter, in this manner. I can ask Rixstep to remove the copyrighted material. Finally, I can inform the audience of this blog about what’s going on.

Scott, via Stepwise, has been instrumental in my journey with Mac OS X programming. When I was first getting acquainted with Cocoa in the OS X Beta days, I often found myself over at Stepwise, checking out the tutorials. When I began writing tutorials on this blog, Scott would link to some of them. I often used Stepwise as a barometer as to how relevant my posts were.

I hope that Rixstep will relent, and remove the offending material. It would be best for the Mac developer community if Stepwise came back.

Overview

Dodge and burn are photographic processes that lighten or darken a specific part of an image, respectively. Dodge works by placing an object between the light and the photo, such that the area that should be lightened is blocked from the light. Burn works the same way, except that the card is placed such that it disallows light to every part of the photo except the one the developer wants to darken. If you’re interested, Wikipedia has more in depth coverage of dodge and burn.

Our dodge and burn tools are simply going to lighten and darken the image. As in most bitmap editors, our dodge and burn tools will act like brushes.

For example, if we have a simple black and white linear gradient, the dodge tool affects it like so:

The burn tool has the opposite effect on the same image:

In addition to being able to adjust the strength, often called exposure, of the dodge and burn tools, the user will also be able to determine the range of colors the tools will affect the most: highlights, midtones, or shadows. Although our dodge and burn tools do not exactly mimic the tools found in large commercial packages, I believe they get reasonably close.

Implementation Overview

Since dodge and burn are brushes, they inherit all the properties of our basic bitmap brush. The difference is, instead of using a solid color for the brush tip, they use the source image, after it has been altered by a Core Image filter.

Since the basic brush implementation has been covered before, we’re only going to cover the basic algorithms used in the Core Image filters for dodge and burn. Both the dodge and burn tools only affect a specific range of colors at a time: highlights, midtones, and shadows. Because of this, there will be six filter algorithms shown here (three for dodge and three for burn).

The filters affect each of the color components in the same way. i.e. The function applied to the red channel is the same as the function applied to the green and blue channels. In addition, the range parameter is also per color component. Highlights are the component values closest to 1.0, while midtones are around 0.5, and shadows are close to 0.0. Finally, we’re not going to cover how to implement the exposure parameter here in the overview, but instead show the general filter functions.

Dodge Highlights

The dodge highlights filter will lighten all components, but will affect the components near 1.0 the most. The function for this filter is:

new component = component + e ^ component – 1.0

where component is one of pixel component values (red, green, or blue) and e is the math constant.

This yields the graph:

As you can see, for component values closer to 0 (i.e. shadow values), the function is close to the identity function (new component = component, aka y = x). As the component values approach 1, the function diverges from the identity greatly in that the new component value reaches 1 much quicker. The result is that components closer to 1 are pushed towards 1 more quickly than they normally would. i.e. Highlights are lightened.

Dodge Midtones

Dodging midtones will lighten all components, but it will affect the components near 0.5 the most. The function for this filter is:

new component = component + 0.25 * sin(component * PI)

where component is one of pixel component values (red, green, or blue) and PI is the math constant. The constant of 0.25 is just to limit the amplitude.

This yields the graph:

As the graph demonstrates, the function intersects at (0, 0), and (1, 1), just like the identity function, but bows upward in the middle, near 0.5. The result is that components near 0.5 are pushed closer to 1, or in other words, lightened.

Dodge Shadows

Dodge shadows, like all the other dodge filters, will lighten all colors, but will affect the components closest to 0 the most. The function for this filter is linear:

new component = 0.5 * component + 0.5

where component is one of pixel component values (red, green, or blue).

This yields the graph:

The function intersects at (1, 1), like the identity function, but quickly diverges from it, as the component approaches 0. As it approaches 0, the new components are lightened, in relation to the identity.

Burn Highlights

Burning the highlights will darken all colors, but will effect the components closest to 1 the most. The function for this filter is:

new component = 0.25 * component

where component is one of pixel component values (red, green, or blue).

This yields the graph:

Note that the function intersects at (0, 0) like the identity function, but then skews the values closer to 1 to closer to 0. i.e. It darkens the highlights.

Burn Midtones

Burning midtones will darken all components, but affects the components closest to 0.5 the most. The function for this filter is almost identical to that for dodging midtones:

new component = component – 0.25 * sin(component * PI)

where component is one of pixel component values (red, green, or blue) and PI is the math constant. The constant of 0.25 is just to limit the amplitude.

This yields the graph:

As the graph demonstrates, the function intersects at (0, 0), and (1, 1), just like the identity function, but bows downward in the middle, near 0.5. The result is that components near 0.5 are pushed closer to 0, or in other words, darkened.

Burn Shadows

The burn shadows filter will darken all components, but will affect the components near 0 the most. The function for this filter is similar to that of dodge highlights:

new component = component + (1.0 – e ^ (component – 1.0))

where component is one of pixel component values (red, green, or blue) and e is the math constant.

This yields the graph:

As you can see, for component values closer to 1 (i.e. highlight values), the function is close to the identity function (new component = component aka y = x). As the component values approach 0, the function diverges from the identity greatly in that the new component value reaches 0 much quicker. The result is that components closer to 0 are pushed towards 0 more quickly than they normally would. i.e. Shadows are darkened.

Code Architecture

As always, I’ve provided sample code for this article. Unlike previous sample code, the code for dodge and burn is somewhat forward looking, in that it provides a framework for future filter brushes. In the future, we can simply discuss the subclasses of FilterBrush and the corresponding filter instead of covering all the other brush related code that just changed a little.

Although all the brushing code is heavily borrowed from the smudge tool, it has been refactored so that it can support generic filter brushes. We’ll go over the changes required to support that, but in general, we’ll ignore the code we’ve already covered in previous articles.

Because the code attempts to set up somewhat of a framework, there are nine classes that make up the sample code: MyDocument, CanvasView, Canvas, GenericBrush, FilterBrush, Dodge, Burn, BurnBrushFilter, and DodgeBrushFilter. Fortunately most of these are carry overs which we’ll ignore since we’ve already dealt with them.

MyDocument, CanvasView and Canvas are the same as they ever were, although Canvas has two new methods which I’ll present later. GenericBrush is all generic brushing code that we’ve carried over ever since the first brush article. It contains a couple of new hooks that subclasses can override.

FilterBrush is where the new code begins. It is an abstract subclass of GenericBrush and deals with rendering a filtered section if the canvas as a brush stamp. Concrete subclasses only need to overload a method to return a configured CIFilter to work.

Dodge and Burn are subclasses of FilterBrush. They load their respective filters, DodgeBrushFilter and BurnBrushFilter, and override a method to return instances to their FilterBrush base class.

DodgeBrushFilter and BurnBrushFilter are CIFilter subclasses, and are nearly identical except for which kernels they load up. They are responsible for actually applying the filter.

We’ll first cover the new methods in Canvas and GenericBrush, then move on to FilterBrush, Dodge, and DodgeBrushFilter. Since Burn and BurnBrushFilter are so similar to their dodge counterparts, we’ll ignore them. However, all the Core Image kernel code will be covered since it is different for each.

Canvas

There are only two new methods on the Canvas class and they’re just used for supporting applying filters to the canvas.

The first one returns the CGContextRef for the canvas:

- (CGContextRef) context
{
	// Just grab the context off the layer
	return CGLayerGetContext(mLayer);
}

Nothing to really discuss here: we just ask our backing CGLayerRef for the CGContextRef.

The next method gets a CIImage that can be used for the input of a CIFilter. Because of a bug, it’s a bit more complicated that it should be:

- (CIImage *) image
{
	// We should theoretically be able to just return [CIImage imageWithCGLayer:],
	//	but we're not able to and get the proper results. Core Image does something
	//	a little indeterminate with the pixels: the effect is applied, but the component
	//	values are shifted.
	// This seems to be caused by the fact we're using a CGLayerRef for both the
	//	source and destination of the effect. I asked on the quartz-dev list
	//	if this was supported or not, and received no answer.
	// The work around is create a deep copy of the layer in a bitmap context.
	//	It is heavy, but it works. Simply created a duplicate CGLayerRef here
	//	doesn't seem to work.
#if 1
	CGSize size = CGLayerGetSize(mLayer);
	size_t width = size.width;
	size_t height = size.height;
	size_t bitsPerComponent = 8;
	size_t bytesPerRow = (width * 4+ 0x0000000F) & ~0x0000000F; // 16 byte aligned is good
	size_t dataSize = bytesPerRow * height;
	void* data = calloc(1, dataSize);
	CGColorSpaceRef colorspace = CGColorSpaceCreateWithName(kCGColorSpaceGenericRGB);

	CGContextRef bitmapContext = CGBitmapContextCreate(data, width, height, bitsPerComponent, bytesPerRow, colorspace,  kCGImageAlphaPremultipliedFirst);

	CGColorSpaceRelease(colorspace);

	CGContextDrawLayerAtPoint(bitmapContext, CGPointMake(0, 0), mLayer);

	CGImageRef imageRef = CGBitmapContextCreateImage(bitmapContext);
	CGContextRelease(bitmapContext);
	free(data);

	CIImage* image = [CIImage imageWithCGImage:imageRef];
	CGImageRelease(imageRef);

	return image;
#else
	return [CIImage imageWithCGLayer:mLayer];
#endif
}

Technically speaking, we should be able to just call the static CIImage method imageWithCGLayer to get our CIImage. Unfortunately, there appears to be a bug in Core Image when you use the same CGLayerRef as both the source and destination of a CIFilter. In that case it tends to lighten the effect a lot more than it should. I am not sure why.

The workaround is to create a bitmap context, copy our layer into it, create a CGImageRef from that, then produce a CIImage from the CGImageRef. It’s a lot more involved, not to mention slower.

GenericBrush

The GenericBrush is the same brushing code that it has always been. In the sample code however, we’ve pulled it out into its own class and added a few methods that can be overridden by subclasses. By default, these methods do nothing:

// Subclass overrides so that they can know when we about to start and stop brushing
- (void) startBrush:(Canvas *)canvas;
- (void) stopBrush;

	// Override for sublcasses to implement brush render
- (void) renderInCanvas:(Canvas *)canvas bounds:(CGRect)bounds at:(NSPoint)point;

startBrush and stopBrush are optional overrides, used in case the brush needs to do special set up or take down before or after a brush stroke. startBrush is invoked inside the mouseDown handler before the first stamp is rendered, and stopBrush is invoked in the mouseUp handler after the last stamp is rendered.

renderInCanvas is a required override for subclasses. It renders a single brush stamp. Before invoking renderInCanvas, the GenericBrush class will have already set up the canvas with the proper clipping and whatnot. The subclass only needs to render a stamp at the specified point.

FilterBrush

The FilterBrush class derives from GenericBrush and provides a base class for all future filter-based brushes, which, in this case, will be Dodge and Burn. It does all the set up for Core Image and the actual rendering of the single brush stamp.

Initialization

Although FilterBrush doesn’t add any new properties to the brush, it does need to cache some data, as seen in the init method:

- (id) init
{
	self = [super init];
	if ( self != nil ) {
		// Initialize to nil
		mFilter = nil;
		mCIContext = nil;
	}
	return self;
}

In order to get decent performance out of brushing with a filter, we have to cache both the CIFilter we’re applying and the CIContext we’re rendering into. If we don’t, performance is bad enough that the brush is unusable at any size.

Setting up the Brush Stroke

In order to implement the caching, we use the startBrush and stopBrush methods we just covered in GenericBrush. This means we cache the filter data for one brush stroke. With some more complicated logic, we could potentially be more aggressive with the caching and get even better performance, by caching for more than one stroke.

First, let’s cover the startBrush method:

- (void) startBrush:(Canvas *)canvas
{
	// This gets called right at the mouse down, before we start stamping. We
	//	should do any set up here.

	// Ask the subclass for our filter. Cache it since it is expensive to keep
	//	creating for each stamp we render.
	mFilter = [[self filter] retain];

The first step is to get and cache our filter. We don’t know what the filter is, being an abstract class and all, but our subclass does, so ask them. We, as the base class, expect the filter to already be configured and ready to go, with the exception of the input image:

	// Ask the canvas for a representation of itself as an image. Pass that off
	//	to our filter to be applied to.
	CIImage* inputImage = [canvas image];
	[mFilter setValue:inputImage forKey:@"inputImage"];

We ask the canvas for a CIImage representation of itself, and then give that to the filter as input.

Finally, we need to cache the CIContext we’ll be rendering into:

	// Since we're a filter brush, we're rendering right back onto the canvas,
	//	so ask the canvas for the context to render into. Convert the normal
	//	context into a Core Image context, and cache it off (it's expensive to
	//	destory).
	CGContextRef context = [canvas context];
	mCIContext = [[CIContext contextWithCGContext:context options:nil] retain];
}

Not much here. We ask the canvas for its context, then build a CIContext from it.

Rendering a Stamp

Now that we have our filter and context set up, we’re going to start getting render requests from the base class. We handle that in renderInCanvas:

- (void) renderInCanvas:(Canvas *)canvas bounds:(CGRect)bounds at:(NSPoint)point
{
	// Render a single stamp. In our case, that simply means asking the Core Image
	//	context to draw the output of our filter, at the specified stamp location.
	//	Our base class that called us has already set up the mask clipping so
	//	our stamp will be properly shaped.
	[mCIContext drawImage:[mFilter valueForKey:@"outputImage"] atPoint:bounds.origin fromRect:bounds];
}

Since we already have the filter and context, and the base class has already set up the context and helpfully passed in the stamp bounds, all we do is ask the CIContext to render the resulting image. Easy as pie.

Tearing down the Brush Stroke

When we are done with a single brush stroke, we need to clean up our cache. We do that in stopBrush:

- (void) stopBrush
{
	// This gets called after the mouse up, and the last stamp is rendered. We
	//	should do any clean up here.

	// We're done with the filter and context, so free them up.
	[mFilter release];
	[mCIContext release];
	mFilter = nil;
	mCIContext = nil;
}

This is pretty self-explainatory: we release the cached filter and context. Note that this is a fairly expensive operation.

Spacing

As we saw with the smudge tool, stamp spacing changes based the kind of brush. The filter tools, dodge and burn, are no different. So we overload the spacing method:

- (float) spacing
{
	// By filter brushes typically want closer spacing so the effect is smoother
	return 1.0;
}

Through trial and error I discovered decreasing the spacing to one pixel improved the quality of the rendering, so we do that here.

Loading an Image Unit Plugin

Although not used in the FilterBrush class itself, FilterBrush does provide a helper method to load up image unit plugins. This is useful for subclassed brushes who have custom filters stored in the application bundle:

- (void) loadFilter:(NSString *)filterName fromPlugin:(NSString *)pluginName
{
	// Helper function to ensure the given filter is loaded, so we can use it.

	// Ask the system to load up all the plugins it knows about
	[CIPlugIn loadAllPlugIns];
	NSArray *filterList = [CIFilter filterNamesInCategories:nil];

	// Check to see if our filter is loaded (it should be if we added to a system
	//	path.). If it is in a plugin in the application bundle, then it won't
	//	be found.
	if( ![filterList containsObject:filterName]) {
		// It wasn't loaded by default, so manually load it

		// Construct the path to the plugin bundle. We assume it's in the application
		//	bundle, in the plugins folder.
		NSString *path = [[[NSBundle mainBundle] builtInPlugInsPath] stringByAppendingPathComponent:pluginName];

		// Explicitly load the plugin, given the path
		[CIPlugIn loadPlugIn:[NSURL fileURLWithPath:path] allowNonExecutable:NO];
	}
}

This is fairly standard Core Image code. We ask the system to load up all the known Core Image plugins, then look for our specific filter. If it’s there, then we’re done. If not, then we look in our Plug-Ins folder inside our application bundle for the plug-in, and ask CIPlugIn to manually load that plug-in.

Dodge

The Dodge class implements the dodge tool by deriving from the FilterBrush, and overriding the filter method. It has a couple of parameters, which are the same as the Burn tool.

Parameters

The two parameters are initialized in the init method (in addition to the parameters in the GenericBrush init):

- (id) init
{
	self = [super init];
	if ( self != nil ) {
		// First, make sure our filter is loaded. We don't have to do it now
		//	but its convienent here.
		[self loadFilter:@"DodgeBrushFilter" fromPlugin:@"Filters.plugin"];

		// Set the default values for our parameters
		mExposure = 1.0;
		mRange = kDodgeRange_Midtones;
	}
	return self;
}

There are two new parameters for the Dodge and Burn tools: mExposure and mRange.

• mExposure Exposure determines how strong the dodge or burn is applied. It ranges from 0.0 to 1.0, where 0.0 means the effect isn’t applied, and 1.0 is where the effect is at its strongest.

  Dodge examples:

  • mExposure = 0.25,
  • mExposure = 0.5,
  • mExposure = 1.0,
• mRange Range determines the range of pixels that the dodge tool is applied to. It is an enumeration that includes highlights, midtones, and shadows. Highlights are those closest to 1, midtones those closest to 0.5, and shadows are those closest to 0.

  Dodge examples:

  • mRange = kDodgeRange_Highlights,
  • mRange = kDodgeRange_Midtones,
  • mRange = kDodgeRange_Shadows,

We use a black to white linear gradient because it represents the entire component range, from shadows to highlights.

The only other thing to note about the init method is that we manually load up our filter, by invoking the parent class helper method.

Creating a filter

The only real task of the Dodge tool is to create the appropriate filter based on its parameters. This is accomplished in one method, filter:

- (CIFilter *) filter
{
	// We need to create and configure our filter here.

	// Pull out our special filter, and set the exposure to be exactly
	//	what was given to us.
	CIFilter * filter = [CIFilter filterWithName:@"DodgeBrushFilter"];
	[filter setDefaults];
	[filter setValue:[NSNumber numberWithFloat:mExposure] forKey:@"inputExposure"];

	// Configuring the range is slightly more complicated because we have to
	//	convert an enumeration to a straight number.
	switch ( mRange ) {
		case kDodgeRange_Highlights:
			[filter setValue:[NSNumber numberWithInt:kDodgeFilter_Highlights] forKey:@"inputRange"];
			break;
		case kDodgeRange_Midtones:
			[filter setValue:[NSNumber numberWithInt:kDodgeFilter_Midtones] forKey:@"inputRange"];
			break;
		case kDodgeRange_Shadows:
			[filter setValue:[NSNumber numberWithInt:kDodgeFilter_Shadows] forKey:@"inputRange"];
			break;
	}

	return filter;
}

Here we simply ask CIFilter for an instance of our dodge filter (not the system one), which we loaded up in our init method. Note that our dodge tool parameters have a one to one mapping with the dodge filter parameters, so we just pass them through unchanged. After we’re done configuring our filter, we return it.

It should be noted that the Burn class is identical to the Dodge class, except that it uses the burn filter instead of the dodge filter.

DodgeBrushFilter

DodgeBrushFilter is a CIFilter derived class that is housed in an image unit plugin that our main application loads. It is a fairly standard filter, whose main task is to select the correct kernel to apply to the image passed in.

Like most CIFilter classes, we load the kernels in the init method:

static NSArray	*sKernels = nil;

- (id) init
{
	// If we haven't loaded up our array of kernels, do so now
	if ( sKernels == nil ) {
		// Look for our kernel code file inside of our bundle
		NSBundle *bundle = [NSBundle bundleForClass:[self class]];
		NSString *code = [NSString stringWithContentsOfFile:[bundle pathForResource:@"DodgeBrushFilter" ofType:@"cikernel"]];

		// We have three kernels in the file: highlights, midtones, and shadows.
		//	The range parameter selects which one we'll use. Cache them.
		sKernels = [[CIKernel kernelsWithString:code] retain];
	}

	return [super init];
}

We keep our kernels in a static variable so we don’t waste time loading them each time. The kernels are all stored in one file, DodgeBrushFilter.cikernel, which is kept inside the Image Unit Plugin bundle. We load the cikernel file, convert it to an array of CIKernels, and cache those off. There are three kernels loaded: one for each range.

Our dodge filter has two custom parameters, which we declare in the customAttributes method:

- (NSDictionary *) customAttributes
{
	// Return the custom attributes, which, in our case, is just the
	//	exposure and range parameters.
	return [NSDictionary dictionaryWithObjectsAndKeys:

		[NSDictionary dictionaryWithObjectsAndKeys:
			[NSNumber numberWithFloat: 0.0], kCIAttributeMin,
			[NSNumber numberWithFloat: 1.0], kCIAttributeMax,
			[NSNumber numberWithFloat: 0.0], kCIAttributeSliderMin,
			[NSNumber numberWithFloat: 1.0], kCIAttributeSliderMax,
			[NSNumber numberWithFloat: 0.5], kCIAttributeDefault,
			[NSNumber numberWithFloat: 0.0], kCIAttributeIdentity,
			kCIAttributeTypeScalar, kCIAttributeType,
			nil], @"inputExposure",

		[NSDictionary dictionaryWithObjectsAndKeys:
			[NSNumber numberWithInt: kDodgeFilter_Highlights], kCIAttributeMin,
			[NSNumber numberWithInt: kDodgeFilter_Shadows], kCIAttributeMax,
			[NSNumber numberWithInt: kDodgeFilter_Highlights], kCIAttributeSliderMin,
			[NSNumber numberWithInt: kDodgeFilter_Shadows], kCIAttributeSliderMax,
			[NSNumber numberWithInt: kDodgeFilter_Midtones], kCIAttributeDefault,
			kCIAttributeTypeScalar, kCIAttributeType,
			nil], @"inputRange",

		nil];
}

As with the dodge brush, the parameters are exposure and range. They mean exactly the same thing they did in the Dodge class.

The last part of our filter is the outputImage method, which actually constructs the output CIImage:

- (CIImage *) outputImage
{
	// The idea is to apply the kernel selected by the range parameter, to
	//	the image passed in. We don't really do any fancy preprocessing.
	CISampler *sampler = [CISampler samplerWithImage:inputImage];
	CIKernel *kernel = [sKernels objectAtIndex: [inputRange intValue]];

	// Just apply the chosen kernel
	return [self apply:kernel, sampler, inputExposure, kCIApplyOptionDefinition, [sampler definition], nil];
}

This is also a reasonably simple method. We create a sampler from our input image, and pull a kernel out of our cached array using the range parameter as an index. We then apply the chosen kernel to our image, and we’re done.

Like with the brushes, the burn filter is identical to the dodge filter, except that it loads up the burn kernels and applies those instead.

Kernels

The real meat of all this code are the kernels. They are straightforward implementations of the functions presented in the overview section, so not a lot of explanation will be given. The only difference is the kernels deal with implementing the exposure parameter, which the function presented in the overview section ignored.

Dodge Highlights

kernel vec4 dodgeHighlights(sampler image, float exposure)
{
	vec4 source = unpremultiply(sample(image, samplerCoord(image)));

	float factor = exposure;

	source.rgb = source.rgb + factor * (exp(source.rgb) - 1.0);

	return premultiply(source);
}

Dodge highlights is almost identical to the overview function presented. Note that we apply the function to all three components (RGB) at once, and the exp() function is e raised to the value passed in. In this case, the exposure is used directly to scale the steepness of the function.

Dodge Midtones

kernel vec4 dodgeMidtones(sampler image, float exposure)
{
	vec4 source = unpremultiply(sample(image, samplerCoord(image)));

	float pi = radians(180.0);
	float factor = exposure * 0.25;

	source.rgb = source.rgb + factor * sin(source.rgb * pi);

	return premultiply(source);
}

Dodge midtones is fairly straightforward. We scale the exposure by a quarter just so we don’t push values to 1 too quickly.

Dodge Shadows

kernel vec4 dodgeShadows(sampler image, float exposure)
{
	vec4 source = unpremultiply(sample(image, samplerCoord(image)));
	float factor = (1.0 - exposure * 0.5);

	source.rgb = factor * source.rgb + (1.0 - factor);

	return premultiply(source);
}

For dodging shadows, we scale the exposure by half so we don’t push everything to 1 at its peak. Also, since we’re using a linear function, we have to make the y intercept dependent on the slope. i.e. Both slope and y intercept are based on the exposure.

Burn Highlights

kernel vec4 burnHighlights(sampler image, float exposure)
{
	vec4 source = unpremultiply(sample(image, samplerCoord(image)));

	float factor = (1.0 - exposure * 0.75);

	source.rgb = factor * source.rgb;

	return premultiply(source);
}

Note that burning highlights is also a linear function. However there is no y-intercept because we want it to intersect at (0, 0). Also note that we scale the exposure by 3/4.

Burn Midtones

kernel vec4 burnMidtones(sampler image, float exposure)
{
	vec4 source = unpremultiply(sample(image, samplerCoord(image)));

	float pi = radians(180.0);
	float factor = exposure * 0.25;

	source.rgb = source.rgb - factor * sin(source.rgb * pi);

	return premultiply(source);
}

Burning midtones is identical to dodging midtones, except that we subtract off the sine wave. Like before, we scale the exposure by 1/4 before using it.

Burn Shadows

kernel vec4 burnShadows(sampler image, float exposure)
{
	vec4 source = unpremultiply(sample(image, samplerCoord(image)));

	float factor = exposure;

	source.rgb = source.rgb + factor * (1.0 - exp(1.0 - source.rgb));

	return premultiply(source);
}

Burning shadows is the same idea as dodging highlights, except that we’ve flipped the function upside down.

Method to the Madness

It took me a long time to figure out the current kernels for each of the ranges. On the surface, dodge and burn sound easy: you just lighten or darken the image. But when I started playing with existing dodge and burn tools, I found that it wasn’t quite that simple.

I started out with simple linear functions for the highlights and shadows. With some tweaks this worked respectably well for dodging shadows and burning highlights, but was way off for dodging highlights and burning shadows. In other implementations these tools pushed components to the extreme (either 0 or 1 depending on the tool) very quickly. It seemed exponential to me, which lead to the current implementation eventually. I started with base 2 instead of e though, on the presumption that 2 was more likely because it would execute faster.

Dealing with midtones took me a while. I knew I needed a curve that would be highest in the middle, but wasn’t sure which function would yield the best results. I actually started out with a gaussian function, but I concluded that was way too complex for anyone to really use for a filter. I then played around with making a curve with smooth step, but I wasn’t happy with the results. I’m not sure why sin wasn’t the first function to pop into my head when I knew I needed a curve, but it turned out to be the one to yield the best results.

I’m not sure if this how the big graphics apps implement dodge and burn, but the results seem reasonably close to me. Unfortunately no one seems to publish how they implemented dodge and burn. As a result, this took me a long time to figure out. I literally started writing this article three times before, but stopped because I wasn’t happy with the results of the tools. I also spent a lot of time in Grapher, Apple’s graphing calculator program staring at the graphs of various functions.

Conclusion

In addition to showcasing some relatively complex brushes, this article introduces a framework that we can build off of to implement other filter brushes. For example: sharpen, blur, and sponge tools.

This is one of the more satisfying articles to write, just because of what it took to implement the dodge and burn tools. Enjoy, and be sure to download the sample code.

I had the pleasure of using Acorn for the past couple of weeks, and I’m quite happy with the direction Gus is taking it. It is focused on staying simple, streamlined, and easy to use.

There are no revolutionary or experimental tools in Acorn, just streamlined versions of the ones you would expect to find in a bitmap editor. The focus is not on glitz and glamour and the application itself, but on staying out of user’s way so get he or she can get the job done. i.e. Acorn is trying to evolve the user interface for bitmap editors.

One of the interface features that I was first struck by was the all-in-one tools window. To be honest, my initial gut reaction was: “What was Gus thinking?” However, after using it for a while, I dig it. It keeps everything together in one place, so after selecting a tool I don’t have to move my mouse to the other side of the screen to configure it. Or to select the layer I want to work on. While the all-in-one window wouldn’t work in the 800lb gorilla with all its panels, it works beautifully in Acorn.

I’m not a designer, so I don’t feel qualified to give Acorn a full review. However, I’ve worked on graphics applications before, and I’ve tried Acorn, and I like it, for all that’s worth. Go take it for a spin.

The intent of the articles and sample code is to increase the public knowledge of how tools in bitmap editors work. It appeared to me that every author of a graphics editor had to go out and reinvent the wheel somewhat, simply because not all the basic tools were publicly documented. The sample code is intended not only to satisfy idle curiosity, but to improve graphics editors, specifically for the Mac. i.e. I’m hoping to see Mac graphics apps with these features, if they don’t have them already.

To that end, the license needs to be permissive, so that it can be used in a wide variety of applications.

Privately, I think of the sample code as public domain. Optimally, that’s how I’d want it. However, because of the copyright laws in the US, it’s very difficult to make it so.

Instead, I’ve chosen the MIT license, which will hopefully be lenient enough:

Copyright (c) 2007 Andrew Finnell

Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the “Software”), to deal in the Software without
restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following
conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.

If you’d like to use another license, just let me know. In general I grant permission to use the code in any application, whether it be open source or closed.

One final note: since I’m writing the articles with the hope that some of the tools will wind up in real applications, I’d like to know if they do. If you use some of the code here, I would appreciate it if you would shoot me an email letting me know. This isn’t required, but I would be grateful.

I originally thought the smudge tool would be rather complex. I was pleasantly surprised that it was quite simple to implement. So, as a bonus, I went ahead and threw in a stamp tool implementation too.

Overview

Your basic smudge tools smears the paint as if you dragged a clean brush across it. For example if you start with simple image below:

then drag the smudge tool from the left to the right (i.e. from the blue to the white), your get:

How much of a smudge you get is determined by how much pressure is applied.

The idea behind a smudge tool is that as you drag a brush through paint, both the canvas and brush swap paint. The brush picks up paint from the canvas then deposits it elsewhere on the canvas. If you’re familiar with bitmap editors, that almost sounds like the stamp brush tool, which it is, so as a bonus we’ll cover how to implement a stamp brush tool.

Algorithm

When researching how to implement a smudge tool, I found that just about everyone has a different implementation. They range from the very simple, to the very complex which are trying to accurately simulate the physical characteristics of a brush dragged across paint. The algorithm that I’ll present here definitely falls on the simple side, but it is reasonably close to what popular graphics editors implement.

The first thing to remember is that the smudge tool is just a special kind of brush. As a result, it is implemented in a very similar fashion to bitmap brush. In fact, if you haven’t yet already read the article “How to implement a basic bitmap brush”, you should do that now. The main difference between the simple bitmap brush and the smudge tool is how it renders a single stamp.

Instead of using a single color in the shape of the brush tip, the smudge tool will use a piece of the canvas, in the shape of the brush tip, as the stamp image. It locates the piece of canvas to be used as a brush image by looking at the last location it rendered a stamp. It grabs the canvas at that point and uses it as the brush tip.

The only other deviation from the usual stamping algorithm is the stamp spacing. For most brushes, the stamps are spaced 1/4 of brush tip wide. However, since the smudge tool takes its brush tip from the previous render location, it requires a spacing of one pixel. If we were to use the standard spacing, we’d end up with a choppy smudge, like so:

That’s it for our simple smudge algorithm. If we wanted something more complex, we could create a separate drawing context for the brush tip image, and have it actually accumulate paint from the canvas. We could also have the brush tip textured, so it picked up paint more in the raised areas of the brush.

Code architecture

As always, I have provided sample code to demonstrate the smudge tool. It is heavily based on the sample code from the brushes article, so we will only cover the differences between the smudge tool and a normal brush. There are four classes in the sample code: MyDocument, CanvasView, Canvas, and Smudge.

MyDocument is derived from NSDocument, and exists only to load up a user selected NSImage and hand it off to CanvasView. It is almost identical to the MyDocument used in the magic wand article, plus it doesn’t do anything terribly exciting, so we’ll ignore it from here on out.

CanvasView is a NSView derived class that renders the canvas to the screen, and catches mouse events to pass off to the Smudge class to handle. It also acts like a moderator between the Smudge and Canvas classes. It is nearly identical to the CanvasView class used in the brushes article, so we’ll ignore it the rest of the article.

The Canvas class represents the canvas the user draws onto. It can render itself into a NSGraphicsContext, and provides drawing primitives to render a simple brush stamp and a line of stamps. Although the stamping algorithm hasn’t changed from the brushes article, the Canvas class does use a different backing implementation and parameters, which warrants a quick revisit in this article.

The Smudge class represents a brush that can be used to smear paint across the canvas. Since it is a type of brush, it is heavily based on the Brush class used in the brushes article. In this article, we will only cover what the differences are. In this code, the smudge tool not only processes the user’s mouse events, it is also responsible from rendering a single stamp onto the canvas.

We’ll tackle the changes in the Canvas class first, then dig into the Smudge class.

Canvas

Unlike its previous incarnation, the Canvas class here is backed by a CGLayerRef, instead of a CGBitmapContext. This allows both greater ease of use when stamping from the canvas onto itself, and better performance. There is no init method on this class because it requires an NSImage, which will be specified later. There is a dealloc method, which simply releases the CGLayerRef.

Initialization

The initialization happens when the setImage method is called by the CanvasView class. It is responsible for creating a CGLayerRef of the proper size and rendering the NSImage parameter into the layer:

- (void) setImage:(NSImage *)image view:(NSView *)view
{
	// First free up the previous layer, in case the new one is a new size
	CGLayerRelease(mLayer);
	mLayer = nil;

	// Next, create a new layer the size of the image. For performance reasons,
	//	we want to base our layer off of the window context.
	NSGraphicsContext* windowContext = [NSGraphicsContext graphicsContextWithWindow:[view window]];
	CGContextRef context = [windowContext graphicsPort];

	NSSize size = [image size];
	mLayer = CGLayerCreateWithContext(context, CGSizeMake(size.width, size.height), nil);

Although in the sample code we never call setImage more than once, we cover that possibility here by freeing up any previously existing layer. Note that we pass in the NSView we’ll eventually render into as a parameter. This is for the purpose of creating the CGLayerRef. The layer needs to be based on a CGContextRef similar to the one that it will eventually be rendered into, in order to get the best performance. To that end, we create grab the NSGraphicsContext for the window the view is inside. We then create the layer based on that and the size of the image.

Next, we need to render the NSImage we got as a parameter into our layer:

	// Pull out the NSGraphicsContext from the layer and focus it so we can
	//	use Cocoa class to draw into it.
	CGContextRef cgLayerContext = CGLayerGetContext(mLayer);
	NSGraphicsContext* layerContext = [NSGraphicsContext graphicsContextWithGraphicsPort:cgLayerContext flipped:YES];

	[NSGraphicsContext saveGraphicsState];
	[NSGraphicsContext setCurrentContext:layerContext];

	// Some images might have transparency, so fill the background with an opaque
	//	white. Real apps would probably do a checkerboard pattern.
	[[NSColor whiteColor] set];
	[NSBezierPath fillRect:NSMakeRect(0, 0, size.width, size.height)];

	// Draw the image, with no scaling
	[image drawAtPoint:NSMakePoint(0, 0) fromRect:NSMakeRect(0, 0, size.width, size.height) operation:NSCompositeSourceOver fraction:1.0];

	[NSGraphicsContext restoreGraphicsState];
}

We’re going to use Cocoa drawing classes to render the image, so we need to go from our CGLayerRef to a focused NSGraphicsContext. Fortunately, CGLayerRef has a method that will return a CGContextRef, and NSGraphicsContext has a constructor that will take a CGContextRef. The first bit of drawing we do is to render a white background. This is for the case of a transparent image. Finally, we render the image into the current NSGraphicsContext, which happens to our layer. Our Canvas object is now initialized and ready to be used.

Drawing the canvas onto a view

After initialization, the first thing we’ll be asked to do in the Canvas class is to render. This is covered in drawRect:

- (void)drawRect:(NSRect)rect inContext:(NSGraphicsContext*)context
{
	if ( mLayer == nil )
		return;

	// Very straight forward: just draw our layer in the given context at 0, 0
	CGContextRef cgContext = [context graphicsPort];
	CGContextDrawLayerAtPoint(cgContext, CGPointMake(0, 0), mLayer);
}

This is almost too trivial to cover. If we have created a layer, we render it into the NSGraphicsContext passed in. The End.

Rendering a line of stamps

As we covered earlier, the stamping algorithm hasn’t changed from the brushes article, although one of the parameters has, and a constant derived from that parameter. Since the line stamping is rather long, we’ll only review the part that has changed, which, fortunately for us, is only the function signature and the first line of code:

- (float)stamp:(Smudge *)brush from:(NSPoint)startPoint to:(NSPoint)endPoint leftOverDistance:(float)leftOverDistance
{
	// Set the spacing between the stamps.
	float spacing = [brush spacing]; // Ask the brush

Instead of passing in an image to be used for stamping, we now pass in the smudge tool. This is because the Smudge class is now responsible for rendering a single stamp. Also, we want the spacing for the stamping to be configurable based on the brush. So instead of computing the stamp spacing here in the Canvas class, we ask the brush for it.

The rest of the method is the same as before.

Rendering a single stamp

The last part of the Canvas class to cover is the rendering of a single stamp, which as noted above, is no longer handled in the Canvas class:

- (void)stamp:(Smudge *)brush at:(NSPoint)point
{
	// Just pass through to the brush to ask it to render. Give it the layer
	//	that backs the canvas, and the point to render at.
	[brush render:mLayer at:point];
}

We just hand the rendering of a stamp off to our Smudge class. We provide it with the canvas layer, so that it can pull pixels from it and render into it.

As you can see, the main ideas of the Canvas class didn’t change, but a bit of the implementation did, in order to make it more flexible.

Smudge

The Smudge class contains most of the interesting code. Like its predecessor, Brush, it tells the Canvas class where to render lines and single points. However, it is also responsible for rendering a single stamp. Since Smudge and its predecessor have so much in common, we’ll only discuss where they differ.

Parameters

The smudge tool has some new parameters that weren’t in the basic brush class. They are initialized in init:

- (id) init
{
	self = [super init];

	if ( self ) {
		// Set the size of the brush. A radius of 10 means a 20 pixel wide brush
		mRadius = 10.0;

		// Create the shape of the tip of the brush. Code currently assumes the bounding
		//	box of the shape is square (height == width)
		mShape = CGPathCreateMutable();
		CGPathAddEllipseInRect(mShape, nil, CGRectMake(0, 0, 2 * mRadius, 2 * mRadius));
		//CGPathAddRect(mShape, nil, CGRectMake(0, 0, 2 * mRadius, 2 * mRadius));

		// Set the initial smudge color, that starts out on the brush. May be nil,
		//	if you don't want a smudge color.
#if 1
		mColor = nil;
#else
		CGColorSpaceRef colorspace = CGColorSpaceCreateWithName(kCGColorSpaceGenericRGB);
		float components[] = { 0.0, 0.0, 1.0, 1.0 }; // I like blue
		mColor = CGColorCreate(colorspace, components);
		CGColorSpaceRelease(colorspace);
#endif

		// The "softness" of the brush edges
		mSoftness = 0.5;

		// The pressure at which to smudge. The more pressure, the more of a smudge
		//	will result.
		mPressure = 1.0;

		// Initialize variables that will be used during tracking
		mMask = nil;
		mLastPoint = NSZeroPoint;
		mLeftOverDistance = 0.0;
		mLastRenderPoint = NSZeroPoint;
	}

	return self;
}

There are two new parameters: mPressure and mColor. Although mColor was also in the brush class, its function is different for the smudge tool.

• mPressure Pressure mimics how hard the user is pressing down on the canvas, and as a result, how much the paint is smeared. It has a range of 0.0 to 1.0, where 0.0 means the paint doesn’t smudge at all, and 1.0 means it smudges a lot.

  Examples:

  • mPressure = 0.1,
  • mPressure = 0.5,
  • mPressure = 1.0,
• mColor Color is used to determine if the brush used for smudging is dirty. i.e. Does the brush start out with some paint already on it? If so, it leaves an initial smudge of the specified color. If mColor is nil, then the smudge tool simulates a clean brush.

  Examples:

  • mColor = [1.0, 0.0, 0.0],
  • mColor = [0.0, 1.0, 0.0],
  • mColor = [0.0, 0.0, 1.0],

The rest of the parameters are identical to those in the regular brush class.

Creating the brush tip

The job of creating the brush tip is a bit different in the smudge tool than the original brush tool. In the original brush we created a full ARGB image in the brush color with the correct transparency. Since our brush image is going to be pixels from the canvas, we just want the brush tip, mMask, to reflect the shape, or transparency, of the brush. So instead of a full color image, we create a grayscale image with no alpha that will act as a mask for our actual brush image. This allows us to impose our brush shape onto any random image.

The new createBrushTip method looks like:

- (CGImageRef) createShapeImage
{
	// Create a bitmap context to hold our brush image
	CGContextRef bitmapContext = [self createBitmapContext];

	CGContextSetGrayFillColor(bitmapContext, 1.0, 1.0);

	// The way we acheive "softness" on the edges of the brush is to draw
	//	the shape full size with some transparency, then keep drawing the shape
	//	at smaller sizes with the same transparency level. Thus, the center
	//	builds up and is darker, while edges remain partially transparent.

	// First, based on the softness setting, determine the radius of the fully
	//	opaque pixels.
	int innerRadius = (int)ceil(mSoftness * (0.5 - mRadius) + mRadius);
	int outerRadius = (int)ceil(mRadius);
	int i = 0;

	// The alpha level is always proportial to the difference between the inner, opaque
	//	radius and the outer, transparent radius.
	float alphaStep = 1.0 / (outerRadius - innerRadius + 1);

	// Since we're drawing shape on top of shape, we only need to set the alpha once
	CGContextSetAlpha(bitmapContext, alphaStep);

	for (i = outerRadius; i >= innerRadius; --i) {
		CGContextSaveGState(bitmapContext);

		// First, center the shape onto the context.
		CGContextTranslateCTM(bitmapContext, outerRadius - i, outerRadius - i);

		// Second, scale the the brush shape, such that each successive iteration
		//	is two pixels smaller in width and height than the previous iteration.
		float scale = (2.0 * (float)i) / (2.0 * (float)outerRadius);
		CGContextScaleCTM(bitmapContext, scale, scale);

		// Finally, actually add the path and fill it
		CGContextAddPath(bitmapContext, mShape);
		CGContextEOFillPath(bitmapContext);

		CGContextRestoreGState(bitmapContext);
	}

	// Create the brush tip image from our bitmap context
	CGImageRef image = CGBitmapContextCreateImage(bitmapContext);

	// Free up the offscreen bitmap
	[self disposeBitmapContext:bitmapContext];

	return image;
}

Note that createBitmapContext actually creates a grayscale bitmap context with no alpha that is filled to completely black. The only real difference in createBrushTip is that we set the color to white instead of a user specified color. In an image that is used as a mask, white means that a pixel is full opaque, while black means the pixel will be fully transparent.

Rendering a single stamp

The real meat of Smudge is where we render a single stamp of the brush into the canvas’s CGLayerRef. The render function starts out like the old single stamp method on Canvas:

- (void) render:(CGLayerRef)canvas at:(NSPoint)point
{
	// Grab the context for the canvas. No matter what, we're going to determine
	//	where the current brush stamp should go, then translate the context
	//	to that position.
	CGContextRef canvasContext = CGLayerGetContext(canvas);
	CGContextSaveGState(canvasContext);

	// So we can position the image correct, compute where the bottom left
	//	of the image should go, and modify the CTM so that 0, 0 is there.
	CGPoint bottomLeft = CGPointMake( point.x - CGImageGetWidth(mMask) * 0.5,
									  point.y - CGImageGetHeight(mMask) * 0.5 );
	CGContextTranslateCTM(canvasContext, bottomLeft.x, bottomLeft.y);

We grab the context off the layer, which is really our canvas, and apply an affine transform to it such that the origin is where we want to draw the bottom left of our brush image.

Next, we want to force the shape of our brush onto our brush image:

	// Our brush has a shape and soft edges. These are replicated by using our
	//	brush tip as a mask to clip to. No matter what we render after this,
	//	it will be in the proper shape of our brush.
	CGContextClipToMask(canvasContext, CGRectMake(0, 0, CGImageGetWidth(mMask), CGImageGetHeight(mMask)), mMask);

We simply apply the image we created up in createBrushTip to the context as a clipping mask.

The last thing we need to do before actually rendering the brush image, is to deal with the mPressure parameter:

	// The pressure of the smudge is a one to one correspondance with the amount
	//	transparency of the brush stamp.
	CGContextSetAlpha(canvasContext, mPressure);

As you can see, the pressure is directly related to how transparent with render the brush image. If the user is applying a lot of pressure, then we render the brush image fully opaque. If the user applies less, then the brush image is rendered more transparent.

Now we’re ready to render the brush image:

	// If this is our first stamp, then we normally don't want to lay down any
	//	ink. That's because if we're smudging with a clean brush, we don't have
	//	any ink on the brush to lay down. Only after the initial stamp will we
	//	have the ink from the canvas to lay down.
	if ( !NSEqualPoints(mLastRenderPoint, NSZeroPoint) ) {
		// Based on the last render point that we keep track of, determine the
		//	source bounds.
		CGPoint sourceBottomLeft = CGPointMake( mLastRenderPoint.x - CGImageGetWidth(mMask) * 0.5,
												mLastRenderPoint.y - CGImageGetHeight(mMask) * 0.5 );

		// We pull straight from the canvas, and render directly onto the canvas. CGLayerRefs
		//	make this easy.
		CGContextDrawLayerAtPoint(canvasContext, CGPointMake(-sourceBottomLeft.x, -sourceBottomLeft.y), canvas);
	}

Notice that we check to see if this our first rendering the brush image in the tracking loop. This if statement works because both the mouse down and mouse up handlers reset the mLastRenderPoint member to NSZeroPoint.

If this is our first time, then there is no previous render point to use as our source image, so we don’t do anything. This matches the real world analog: until we move the brush, no smudging occurs.

If this is not our first time trying to stamp the brush image, then we actually stamp the image. We determine the bottom left coordinate of the previously rendered stamp, then use that as our source in the CGLayerRef. Since we have already applied our mask, the pixels from the layer are shaped just like our brush. As you may note, CGLayerRef makes it nice and easy to draw back into itself.

However, if this is our first time trying to render, that doesn’t necessarily mean we have nothing to do:

 	 else if ( mColor ) {
		// If this is our first stamp, and we have an initial color specified (i.e. the brush
		//	was dirty), we have to render the brush with that color only on the first
		//	stamp. The initial color might be carried to other pixels depending on
		//	how strong the pressure is.
		CGContextSetFillColorWithColor(canvasContext, mColor);
		CGContextFillRect(canvasContext, CGRectMake(0, 0, CGImageGetWidth(mMask), CGImageGetHeight(mMask)));
	}

This is the else clause for the if shown above. If this our first time rendering and we have a color specified, then we need to render it. This actually works the same as a normal brush would. We set the fill color to the same as the initial color and fill the entire rect. Since we have a mask applied, only the pixels in the brush shape are drawn.

Note that we only render the initial color on the first stamp. It will get smudged across the canvas on subsequent stamps, since the stamps are only one pixel apart.

The final part of rendering a single stamp is cleaning up, and remembering where we just stamped:

	CGContextRestoreGState(canvasContext);

	// Remember our last render point, so we know where to pull from
	mLastRenderPoint = point;
}

We update mLastRenderPoint so we can use it the next time we stamp.

The last little bit of the Smudge class that we need to examine is the spacing method:

- (float) spacing
{
	// Smudge has to be spaced by 1 pixel or we get jaggies
	return 1.0;
}

If you recall, the Canvas class calls back into the smudge tool to determine how far apart to space the stamps. In order to avoid a really choppy smudge, we return a spacing of 1 pixel.

Stamp tool

We haven’t really spoken about a stamp tool, but if you know what one is, you probably realized that it sounds a bit like the smudge tool in terms of implementation. Briefly, a stamp tool takes a location on the canvas and copies it to another point on the canvas. It is used to copy parts of an image exactly. The source pixels are determined by the user specifying an offset from the cursor for the stamp tool to pull from.

It will probably make more sense when we look at an example implementation. It is exactly the same as the Smudge class except that the spacing and render functions change.

First, the render function for a stamp tool:

- (void) render:(CGLayerRef)canvas at:(NSPoint)point
{
	// Grab the context for the canvas. No matter what, we're going to determine
	//	where the current brush stamp should go, then translate the context
	//	to that position.
	CGContextRef canvasContext = CGLayerGetContext(canvas);
	CGContextSaveGState(canvasContext);

	// So we can position the image correct, compute where the bottom left
	//	of the image should go, and modify the CTM so that 0, 0 is there.
	CGPoint bottomLeft = CGPointMake( point.x - CGImageGetWidth(mMask) * 0.5,
									  point.y - CGImageGetHeight(mMask) * 0.5 );
	CGContextTranslateCTM(canvasContext, bottomLeft.x, bottomLeft.y);

	// Our brush has a shape and soft edges. These are replicated by using our
	//	brush tip as a mask to clip to. No matter what we render after this,
	//	it will be in the proper shape of our brush.
	CGContextClipToMask(canvasContext, CGRectMake(0, 0, CGImageGetWidth(mMask), CGImageGetHeight(mMask)), mMask);

	// Based on the user specified offset, determine the source bounds.
	CGPoint sourcePoint = CGPointMake(point.x + mOffset.x, point.y + mOffset.y);
	CGPoint sourceBottomLeft = CGPointMake( sourcePoint.x - CGImageGetWidth(mMask) * 0.5,
											sourcePoint.y - CGImageGetHeight(mMask) * 0.5 );

	// We pull straight from the canvas, and render directly onto the canvas. CGLayerRefs
	//	make this easy.
	CGContextDrawLayerAtPoint(canvasContext, CGPointMake(-sourceBottomLeft.x, -sourceBottomLeft.y), canvas);

	CGContextRestoreGState(canvasContext);
}

This should look really familiar. We set up the layer’s context exactly the same as the smudge tool, up to and including setting the brush tip as a mask. A stamp tool doesn’t have a pressure parameter, so we eliminate the line that monkeys with the alpha. Since we’re doing a straight replicate with the stamp tool, we always draw from the canvas layer as our source. The only difference is we don’t pull from our last rendered point. Instead we take our current point and apply the user specified offset to determine what we should use as our brush image.

The only other modification we have to make is to the spacing function:

- (float) spacing
{
	// Standard brush spacing
	return CGImageGetWidth(mMask) * 0.25;
}

All we’re doing here is returning the stamp spacing to the normal spacing. We don’t need the one pixel spacing of the smudge tool to avoid a choppy render, and a wider spacing gives us better performance.

The code for the stamp tool is not included in the downloadable sample code, but you should be able to copy paste the methods provided here into the Smudge class and have them work.

Conclusion

The smudge tool was surprisingly simple to implement. I had done a stamp tool previously, but not presented it, because I felt it was too simple to stand on its own. However, given that its implementation is similar to the smudge tool, I jumped at the chance to present it here.

There’s still plenty that could be improved on here. The smudge tool is begging for pressure sensitivity from a tablet. It could also be made more realistic by accumulating paint and using a textured brush.

Happy smudging!

Download the sample code