Sunday, October 18, 2009
Thursday, October 08, 2009
@$(ECHO) -n compiling debug foo.cpp...
@$(CXX) $(CFLAGS) -c foo.cpp -o $@ 2> temp.log || touch temp.errors
@if test -e temp.errors; then $(ECHO) "$(ERROR_STRING)" && $(CAT) temp.log; elif test -s temp.log; then $(ECHO) "$(WARN_STRING)" && $(CAT) temp.log; else $(ECHO) "$(OK_STRING)"; fi;
@$(RM) -f temp.errors temp.log
Friday, October 02, 2009
Tuesday, September 08, 2009
Saturday, September 05, 2009
But fate was a bitch that day... for when I went to try it on VC++ I found that Microsoft had a fairly hokey implementation of Variadic Macros. The basic problem was this, VC++ expands __VA_ARGS__ after macros get expanded which is not consistent with the way the C99 defines the behavior.
#define MY_GREAT_MACRO(...) myGreatFunction(__VA_ARGS__)
Now in GCC that would expand to something like...
MY_GREAT_MACRO(foo,bar) -> myGreatFunction(foo, bar)
But in VC++ you would get something like...
MY_GREAT_MACRO(foo,bar) -> myGreatFunction(foo,bar,)
...notice the trailing comma? Basically what happens is VC++ treats __VA_ARGS__ as a single value and then is expanded as the last step in the preprocessor.
So, basically VC++'s implementation of Variadic Macros is completely worthless... its no different than the old school trick of doing MY_MACRO((my,great,args)).
Microsoft meanwhile doesn't appear interested in fixing this pretty much ever... And yes I realize this is technically a C99 feature and not a C++ feature, but it still doesn't work in C in Visual Studio, and its quite reasonable to expect C features to work in C++ IMO. Microsoft also obviously did attempt to put variadic macros into VC++ so they can't hide behind the "well its a C feature not a C++ feature" bullshit, because they did try and implement it, and they did document it!
Here is a little story of a little library I wrote so that I could load up game assets lighting fast. If you don't like stories you can just download it from here right now instead.
A while ago I setup this fancy serialization tech inside of my hobby engine that allowed easy serialization of arbitrary game classes with virtually no extra effort from the author of said class. The first backend file format was XML simply so I could debug it and it was fairly easy to map a class hierarchy to but always knew eventually I would have to pack the data in a more efficiently... enter libbom, which is a dirt simple bit of code for saving and loading arbitrary data to disk and like XML maps well to hierarchical data structures.
Now before I went and wrote this code I poked around on google for a while to see if I could find if someone else had already done this for me and while I found a bunch of various projects nothing really fit my requirement of being able to load and traverse binary data without reinterpretation. In other words, I ideally wanted to do one big 'fread' and then start traversing my data. I didn't want a lot of memory allocations or converting offsets to pointers.
Elements and Attributes
I tried to model the terminology here as close to XML as possible. So, simply put, an Element contains a Name, a list of Attributes and a list of child Elements. An Attribute contains a Name and some user specified Data.
All data, including the names of Elements and Attributes are stored in tables, and there are two tables.
1) ASCII String Table - this is primarily used for storing Element and Attribute names, but in reality you can store anything you want in there (but its ascii so its probably not useful for human readable strings used in-game). For instance in my hobby project I store all sorts of class and type names inside the BOM string table because it automatically removes duplicate strings. For Unicode strings you will need to use the second table... Also note that strings can be optionally used here, in my engine I only use the strings when the file version doesn't match the engine version.
2) Data Table - this table is not interpreted at all by libbom. The application can simply allocate a block of this table and store what ever they want in it. Data is stored separately from the DOM Tree so that it can be traversed efficiently as well as gives the application a chance to share data between Attributes (the application could, if it wants, remove duplicate entries in the Data Table in order to reduce file size and just reference the same data multiple times).
libbom can read data and traverse it without any conversion process. Basically pipeline is something like this:
1) reads a small header that indicates how much data is to be read, and a few statistics like how many elements vs attributes.
2) allocates some memory.
3) reads the rest of the file into said memory.
4) Profit! you can now directly query the data tree!
Now because libbom was optimized for fast loading the writing phase is not so fast (but not so slow either). Basically libbom contains two parallel sets of classes, the uncooked classes (which are connected by pointers) and the cooked classes (which are contiguous in memory and connected by offsets). To make life as simple as possible to write a file you simply setup a DOM tree using the uncooked classes and then call the 'cook' function to bake them into a single contiguous block of memory which can be written to disk.
libbom does not contain any compression support built in at all... it is assumed the application will either put BOM files together into a single compressed archive or compress them individually itself. Building in compression I do not believe would be of much benefit since many applications will have their own scheme and doubly compressing things does you no service.
So, there you have it, it's simple, fast, and non intrusive. Now I have only tested this on a hand full of platforms but thus far it appears to work on both little and big endian machines and the code is do darn simple that it really shouldn't have any problem on any mature C++ compiler. Let me know if you try it out and what you think. And more importantly, let me know if you know of a better solution out there!
Friday, September 04, 2009
(Blogger scaled the image down slightly so click to enlarge)
Recently I discovered that Sean Barrett has put up on his website a nice little True Type font rasterizer. The keyword here being "little", compared to FreeType its microscopic with less than 1600 lines of source code in a single file. FreeType on the other hand is about 6.5MB of just source code... I have worked on games with less source code! So the possibility of tossing out that much bloat is really appealing to me.
Anyways upon finding this I quickly tossed it into my hobby project game engine to compare performance/memory/quality to FreeType with the assumption that it is probably faster, probably thrashes memory less and quality is probably roughly the same. Here are my discoveries:
This was a big shocker to me... FreeType was a good 3 times faster than stb_truetype at rasterizing a font of about 350 characters at height=14 pixels 8ms vs 26ms... thats almost double a single frame's budget! Granted usually you rasterize fonts once during loading and if you only have 1 or 2 fonts this is probably a non-issue, but if you have lots of fonts and supporting asian localizations this could potentially add seconds to your load time and that might be a deal breaker if you are already barely squeezing into TRC load time requirements. If you are dynamically caching fonts (like because you support rich text or something) this could be make it quite difficult to keep performance up. Luckily in my situation though I am pre-baking my fonts in my tools chain not at load time so performance isn't quite a deal breaker but I would prefer not pre-bake if performance was good enough. One thing to note is that in stb_truetype I was able to set it up to rasterize directly into locked texture memory whereas FreeType has no such API so I have to rasterize and then blit into texture memory so in a way FreeType was given a small handicap already.
Because I rasterize glyphs one by one the peak memory consumption for both FreeType and stb_truetype is quite low (just a few KB). But the number of allocator calls for both is quite high. Oddly enough again stb_truetype hammers on the allocator more making ~8,000 allocations versus FreeType's ~3,000 for the same font. Neither particularly makes me happy as a runtime solution. But the good news here is that while I would not want to touch FreeType's source code with a 10 foot pole, looking around in stb_truetype's source I believe I could stick in a relatively small fixed size pool and make stb_truetype run without any allocations of its own. So there is actually quite a bit of hope here.
Quality of stb_truetype isn't terrible but its not great either. FreeType comes off much cleaner and more closely matches the way Windows rasterize the same font. stb_truetype always comes off a bit bolder than it should be, a bit blurrier and a few artifacts like some characters being 50% transparent. Overall I think quality is probably acceptable for games and tools, but I wouldn't be shocked to have a few artists complain about it looking different than it does in Windows.
Mikko Mononen brought up www.codinghorror.com/blog/archives/000884.html. I am not so sure yet if the differences are within the same threshold or not, but I suppose its possible the quality is no more different than OSX vs Windows (which, btw I think fonts look better in OSX).
I am not quite ready to toss out FreeType yet, but I am awfully tempted. I think all the issues with stb_truetype can be resolved without too much effort so I fully expect to be giving FreeType the boot eventually.
Type 2) Everything you do should be done to make the next thing get done faster.
In a race to see who can get stuff done fastest... Type 1 will win every time when measured on a single task. But if you measure over and extended period of time to see who gets the most done Type 2 will win every time.
It is always a challenge for management to differentiate between the two types because most of the time they measure in such a way that is beneficial to Type 1, but what they really want is a Type 2.
That will conclude today's philosophy lesson.