May 24, 2011

Making-of: Milk 2.5k

This is an attempt to describe a bit about how we made the 2.5k intro, “Milk” by The Compo Xaviours. There’s alot more that went into the intro than I’d like to admit, and I feel that should be documented here. But before I begin, there’s a bit of backstory to be understood.

Going back a bit

Late last December, Decipher and I had plans to complete a highly massive 64k intro for The Gathering. Around January, we started on tool/framework development, and had a pretty good start. We made one mistake, however: We assumed that once we had the technology, the creative aspect of the intro would be trivial. Because of this, we developed highly capable tools and such, but in the end had nothing to show. In other words, we failed :) . On a side note, I was also busy preparing to move to Norway to work for a fantastic startup company founded by former demoscene group Outracks and finish up my current semester at WIT, and Decipher was tied up in work projects. Excuses, excuses ;) .

The original plan

Needing a backup project to avoid arriving at TG empty-handed and deal with the stress I had encountered, I decided in the last three weeks to produce a 1k intro. Problem was, the version of Crinkler 1k that we had on-hand was an older test version that wasn’t very compatible yet (and do to mistakes we had made the previous year, we didn’t want to request a new version unless we absolutely had to). So, I wanted to produce a prototype of Juicer; a 1k compressor that Decipher and I had planned to produce many times before but never got around to finishing.

Making it smaller

So, [re-]development of Juicer began. It started with producing some sort of dumbed-down version, as a proof of concept (and to refresh my very dated data compression experience). For this, I chose to develop a packer for Commodore 64, simply because the executable format is much simpler (no header besides a 16-bit load address, as opposed to the Windows PE header, which is a real pain), and I just love coding for that system. In about five hours or so composed of researching compression techniques, coding, and testing, I had a working lz77-based C64 intro packer. The compression wasn’t very good, but the decompressor (including autostart stub) occupied only 87 bytes, without optimization. Here’s how that looks:

C64 Compressor Prototype

So, it worked. Next was to do it in Windows. If you’ve any of my personal blog posts (like this one), you’ll know I’ve had plenty of trial-and-error experience with PE hacking, so this part was nothing new :) . Once I had the systems (and virtual machines) to test on, I had a working, handmade PE header in assembly in a few hours. The compressor was much harder, though.

The first thing I tried was based on the C64 lz77 scheme, but with some lzma-based modifications. I had decent results on ASCII text (namely, size-optimized GLSL code), but it didn’t work very well on binary data. So, I read up on what Crinkler was doing. As described in the Crinkler manual, this blog post, and a few other sources, Crinkler is doing a (very strange, mind you) 33-bit arithmetic encoder with order-8 context modelling. These buzzwords are too complex to explain here (and that’s what Google’s for, you know), so I won’t :) . But basically, I delved into these methods myself….and I can’t yet share my results until the project’s finished, so let’s move on a bit ;) .

These buzzwords are too complex to explain here

Long story short, three weeks simply isn’t enough (at least for me, hehe) to beat such an efficient compressor as Crinkler 1k with a decent decompressor size. Not to mention I lost two days from TG stuck at the airport in Boston, but that’s another story.

Call to action

So, we had nothing to show at The Gathering, except perhaps a nice effect I planned to squeeze in the 1k. Which I was going to save until we did a proper packer, until Duckers/Outracks called us 30 minutes before the intro compo saying, “If you don’t release something, we have to drop the compo because of the number of entries.” So, with 30 minutes to do an intro, you do what you can with what you have :) .

And, well, you’ve seen the result: Milk by The Compo Xaviours.

It’s the 1k content I had planned, with Decipher’s C framework and a quick tune with my 4k synth. The visuals are kindof cool, so I’d like to break them down for you. Here is the code:

Vertex shader:
varying float t2;
void main(){
t2=gl_Color.x*2222222;
gl_Position=ftransform();
}

Fragment shader:
varying float t2;
float t=t2-fract(t2*.4),d,c=cos(t*.6)*4;
vec3 r(vec3 x,float t){
for(float v=0.;v<3.;v++)t*=.8,x=vec3(cos(t)*x.x+sin(t)*x.z,cos(t)*x.z-sin(t)*x.x,x.y);
return x;
}
void main(){
vec3 x,a,z=vec3(0),y=r(normalize(vec3((gl_FragCoord.xy-vec2(320,180))/300,1)),c);
for(float v=0.;v<3.;v+=.06){
x=abs(a=r(abs(z-r(
vec3(sin(t*.4),0,3.5+sin(t)+t*.05)// Camera
,c))
-vec3(clamp(t-20,0.,1.5))// Mirror displacement
,
max(t-10,0.)*1.2// Mirror rotation
));
d=min(max(max(x.x,x.y),x.z)-1,8.-length(x));
for(float v=0.;v<3.;v++){
x=abs(fract(a*pow(3.,v))-.5);// Subdivision
d=max(d,mix(
.17-min(min(max(x.x,x.y),max(x.x,x.z)),max(x.y,x.z)),// Cubes
length(x)-.6,// Spheres
.5-cos(t*.07)*.5
));
}
gl_FragColor=mix(vec4(1.1,1,1.2,1),vec4(1,1.1,1.2,1),y.y+.5)*v*.3;
if(d<=0)break;
z+=y*max(d,.0001);
}
}

..slightly unobfuscated, that is. Basically, it’s the popular sphere tracing technique, but with some neat little tricks. Let’s break some of it down. We start with this:

This image really only has two objects; a cube and a background sphere. These just look like this:

Then, I intersect the two volumes with a repetitive cutout cube. This looks like this:

Simple enough. Now, do this with multiple iterations, where you decrease the size of each cutout and increase the frequency, and you have the original image. The next trick is the cube deformation, seen here:

This is MUCH simpler than it looks, actually. Let’s remove the “sponginess,” for clarity:

In this shot, it’s actually quite obvious what it is that I’m doing. I’m tracing just the cube and sphere normally, except for one thing: mirroring. Imagine 3 mirrors, set on each axis, and all facing the upper-right corner. This is the entire trick. Everything you see is just a mirror of the upper-right corner (1st domain). So, this deformation is just a matter of rotating the cube. That’s all :) . Camera operations still work normally, so it looks quite nice.

Smoke and mirrors

The next trick, if you look at the original screenshot of the prod (with the four spherical things), is that the cube separates into four more cube-like structures. This is actually the same trick as before; the mirrors. All I do is move the cube outwards diagonally, and the mirrors follow. The demoscene really is all smoke and mirrors, anyways ;) .

My next trick explains why the objects in the main screenshot are spherical. This is simple, too. As the intro progresses, the cube cutouts actually transform into spherical cutouts. And, well, that’s actually enough explanation :) . Instead of cutting out cubes, we cut out spheres. Blend from cube to sphere, and you have the main timeline for the intro.

Finally, we have the “multiple camera angles.” Actually, we don’t :) . This is the simplest part of the intro, and one of the most rewarding. Instead of cutting multiple cameras, we just create one that’s very dynamic. After that, just cut time instead. The result looks like multiple angles, when in reality it’s multiple instances of time.

Wrapping up

So, that’s it for the intro. The 1k version uses some more nasty hacks to get it smaller, but the posted code (obfuscated, of course) is exactly what made it into the actual prod. Had it been 1k, I think I’d've gladly used the Youth Uprising label, but it was just too insignificant as a 2.5k I think :) . All in all, I think it was a good excercise in recursive fractal generation and size-optimizing shader code, and it gave me an excuse to play around with compression for awhile. I think, though a relatively simple release, it was one of my favorite projects to work in in a long time.

Until next time!

avatar
About the author, Jake Taylor

Jake is a coder, musician, engineer, and is absolutely obsessed with the demoscene and everything in it. He has developed many kinds of demos on many kinds of platforms, as well as dabbling in other fields such as circuit design and painting. Though American-born in the wilderness of Idaho, he currently lives and works as a real-time graphics engineer for Outracks Technologies in Norway.

3 Comments Post a comment
  1. avatar
    Mathias Tangen Leganger
    Aug 23 2011

    Awesome :)

  2. avatar
    Ebbe Berge Smith
    Aug 25 2011

    A very cool read! Inspiring stuff. Great display of talent mixed with healthy dose of craziness (the good kind) :)

  3. avatar
    Aug 25 2011

    haha, thanks!

Leave a Comment