August 19, 2011
Node Code: Quartz Composer Overview (Part 1 Of Infinity)
This is going to be an ongoing column, concerned mostly with execution graph technology and Quartz Composer. I originally was going to write a piece about Quartz Composer, and attempt to cover basics and some uber-cool projects done with it. The reality is that the subject is way too broad to cover in one article.
Since I’ve become involved with Quartz Composer (also known as “QC”), I’ve read countless posts to Apple’s Quartz Composer Developer list or Kineme where users complain about a lack of documentation. I don’t think that criticism is entirely valid; I think it’s a matter of history, and this article is going to go into that a bit. What follows will be a bit of broad overview of some issues surrounding Quartz Composer, while future articles will be more in depth and certainly have a great amount of info that one can use in many domains, not just Quartz Composer, though there will be a focus on implementing in QC.
Back to the common complaint of lack of documentation; a cursory browsing of the internet will reveal countless pages that attempt to explain Quartz Composer, they just don’t explain the problem the user wants to solve. Users often disregard Apple’s documentation as a starting point as well. I suggest Apple Quartz Composer Programming Guide as a very good piece of reference material.
Apple has done a great deal of documenting of Quartz Composer, provided tons of example projects that show how to work with Quartz Composer graphs (qtz files), how to write plugins for Quartz Composer, or integrate it into applications that use the technology. Doing a search through Apple’s Developer code will reveal numerous examples and articles, all of which I would recommend that anyone interested in Quartz Composer examine. I think these are the absolute best examples and documentation of Quartz Composer in existence.
The documentation workaround
That said, there is a very major reason why people tend to immediately ignore Apple’s documentation, and it’s somewhat valid.
With Apple’s standard API, one can indeed expand the modules that Quartz Composer loads into it’s environment by writing plugins that one can construct graphs with, but it also places numerous embargoes on that QCPlugin API. There is a large subculture – or perhaps the dominant culture – of QC users that utilize what has become known as “the SkankySDK” to circumvent this. For example, Apple may prevent plugins from loading inside of certain environments like the Web Browser, or from utilizing built in QC data types like “QCMesh” (because of proprietary reasons or simply a lack of updating the spec. Speculation is probably unhealthy!)
Before Apple had actually published a plugin API, users were eager to extend Quartz Composer’s functionality. QC was based on Pixelshox, which was also an extendible technology, and number of users had noted that Apple had also designed Quartz Composer to load “GFPlugins”. By backwards engineering this API, this soon became a way for users to extend Quartz Composer’s functionality. By the time that Apple had actually introduced a plugin API, several developers were already using the “skankySDK”, and numerous plugins that conformed to this standard “non-standard” had been released. Interestingly, Apple continues to create GFPlugins for use in their own projects, choosing not to eat their own dog food. The infamous tape where Apple calls the reverse engineered SDK the “skanky” SDK while introducing their hobbled SDK is entertaining enough though.
What was most problematic is that through use of GFPlugin/skankySDK one could create new data types inside of Quartz Composer, and leverage control over the entire system. QCPlugin uses a wrapper that precludes one not only creating new data types, but also creating certain types of macros, or even utilizing certain built in data types.
While it is far from useless, and one can create a great deal of things in that box, the fact that QCPlugin is limited, but accepted by Apple, has a significant impact on Quartz Composer development and culture. If one wishes to create new modules that are allowed in apps to be sold on Apple’s app store, one must write plugins in that box. If you wish to write modules in a way that leverages complete control, you use the skankySDK, fully knowing that this isn’t viewed as an awesome thing by Apple, and also, that they do not support.
With a little help from your friends
Over the years, Kineme codified the non-standard plugin API/skankySDK into an actual framework, which is available at their site. Kineme’s forum is also the largest community of Quartz Composer users, and a great place to get feedback and help, as well as an excellent source of add on modules. Another very noteworthy developer in the QC community is Anton Marini, a.ka. Vade. His v002 plugins are considered to be some of the most useful plugins available for Quartz Composer, and he also shows how far boundaries can be pushed even when using Apple’s standard QCPlugin API (and he also has a forum for support).
It may seem quirky to lay all of that out on the front end, before even getting too far into “what Quartz Composer is”, but I never said I wasn’t quirky. I think it’s important to understand this when delving into Quartz Composer, and good to understand it before getting deep into interacting and programming with the system.
The inner workings of QC
Quartz Composer is multifaceted. It presents itself in the form of a framework for OS X, as a technology, and as an application to leverage the technology. A good way to think of Quartz Composer is that one is creating a file, and that by linking together modules (patches) with cables (noodles), one can send information from one module to another. Anyone that is familiar with node based editing systems may also think of these as nodes, and of Quartz Composer as a node graph, and that’s an accurate description. Others may think of this as a decision graph, or an execution graph. There are strong parallels with non-programmer paradigms, especially signal flow. Musos that have chained together multiple effects boxes, or modular synths have a good running start to understanding the method of interaction with QC. Users typically work with the Quartz Composer.app, which is included with Apple’s Developer Tools, to create and manipulate Quartz Composer graphs.
In Quartz Composer, a method is used that is called “lazy evaluation”. QC graphs tend to lay out from right to left on the Quartz Composer editor application. When one connects the output of a QC Video Patch, to the image input of a QC Sphere, execution happens thusly: The QC Sphere is a “Consumer” patch. First, a check happens to see if it is enabled. Then, all input port values are surveyed.
If there is a value at the port, it’s used. If there is a connection to another patch upstream, like the Video Patch, that patch then executes. If the Video Patch output was connected to nothing, it would never execute at all. If there was a switcher in between (known as a multiplexer in QC parlance), and connections existed, but the Video Patch was not currently chosen, the Video patch would still not execute. If that Video Patch has an input port, connected to another patch’s output, then it will trigger the next patch in the line to execute. This is likely an important distinction to make to people familiar with graph environments that use a Push model (like Max or VVVV).
I’m excited to delve into that more. In coming installments, we will learn about creating graphs, prototyping shaders and CL kernels, extending functionality with the standard and skanky SDK’s, integrating Quartz Composer into standalone apps, as well as broader issues in coding and how they interplay with this technology.