The Wengerd Report: The meaning of Q?+:$$ &9*^0E#1@2AF5+_R)!/&#<*

by Owen Wengerd

Handles are those 16-digit unique keys attached to every entity within an AutoCAD drawing file. They are commonly exposed and displayed as strings, but they are really 16-digit (64-bit) hexadecimal numbers ranging from zero to more than 1.8e19. That's 18 quintillion for US readers (18 trillion for the rest of you). Or if you prefer, that's just under 1/5 of a googol. No matter how you count it, that's a lot of handles!

Handles are never recycled and persist in a drawing file for the life of the entity, so every new handle must be guaranteed to have never before been used in the drawing. To guarantee this, every drawing file contains a handle seed whose value never decreases -- but increases by one every time a new handle is used. Erasing an entity does not free its handle for reuse; a copy of an entity is always assigned a new handle.

There's a fun fact about the history of handles. Prior to AutoCAD Release 13, handles could be turned off. This saved space in the drawing file, but it also meant there was no getting the original handles back once they were gone. The Handles command could be used to destroy handles in an open drawing file, but the command made you correctly answer a randomly selected question before it did the deed -- just to make sure you wouldn't destroy handles accidentally. If you were too lazy to answer the question, you could cheat and enter the secret master password instead:

Q?+:$$ &9*^0E#1@2AF5+_R)!/&#<*:

Handles are specifically designed to establish and maintain links between drawing entities and data in external databases, or between entities of the same drawing file. Handles can also be used for forensic analysis of drawing files. The gaps between handle numbers, their magnitude, and their order provides valuable information in courtrooms to paint a picture of the drawing's history. For example, handles can prove that two drawing files had a common ancestor, or that one drawing was copied from another.

Many data formats invented in the 1970s and 1980s used data fields that were no more than 32 bits wide, on the assumption that four billion possible values were plenty. But the set of possible handles in AutoCAD is sufficiently large to easily suffice on today's computers. Whoever invented DWG handles had the foresight to make them large enough that they would, for all intents and purposes, last forever. 

But will they?

It's not surprising for software to fail unexpectedly when it reaches internal data size limits or by exceeding itself in unanticipated ways. A little experimenting by manually editing the $HANDSEED value in a DXF file shows that in certain special cases AutoCAD 2011 will crash when the handle seed is set to a value near the maximum, forcing it to recycle handles by wrapping the handle seed back to zero. This is the eNullHandle error. In fact, AutoCAD 2011 will display the following error message when opening a DXF file whose handle seed is merely half way to the theoretical limit: DXF file's HANDSEED '7FFFFFFFFFFFFFFF' is too low. Please edit and change it to '0'.

It's easy to assume that limits as insanely large as multiple quintillions will never be reached, but every finite limit is destined to be reached eventually. Today it may be unimaginable for a drawing file to contain trillions of entities, but it is only a matter of time before we start modeling entire cities, or the entire world, or even the universe itself.

On the one hand, it's no big deal, for handles can be expanded when we start approaching their limit. On the other hand, I predict that CAD users will begin to see more and more limitations due to data field sizes causing problems over the next decade, as CAD drawings expand beyond the 32-bit world.

[ Owen Wengerd writes on programming issues at his blog, Outside the Box. He is president of ManuSoft and vp Americas of CADLock, Inc. ] 

Comments

[Emmanuel Garcia]

I wonder how AutoCAD tracks old handles no longer used because of deleted entities. Does it leave a mock / dummy entity, keep a list or use yet another method? This is a nice informative article. Thank you for writing it.

On a similar vein, since you mentioned forensics, I wonder what the "required qualifications" are for expert testimony, as this is a highly specialized niche. I imagine some of the best and most experienced hackers were too busy to be studying Computer Science 495 in a traditional school.

Warm regards from Los Angeles,

Emmanuel

[Owen Wengerd]

@Emmanuel:

AutoCAD doesn't keep track of discarded handles per se; it uses the handle seed value to ensure that all new handles are larger (and therefore different) than any previously used handle.

Although it certainly doesn't hurt, an "expert" doesn't need a degree to be considered an expert. I've never testified directly as an expert witness, but I've worked as an expert consultant in several lawsuits involving DWG files -- and I only have a high school education.

[Julian Hardy]

"They are commonly exposed and displayed as strings, but they are really 16-digit (64-bit) hexadecimal numbers ranging from zero to more than 1.8e19. That's 18 quintillion for US readers (18 trillion for the rest of you). Or if you prefer, that's just under 1/5 of a googol."

Ummm ...

1 googol = 1.0e100 (1 x 10^100)
1/5 of a googol = 2.0e99 (2 x 10^99)

1.8e19 is less than 1 part in 10^80 of 1 googol.

For all practical intents and purposes, 1.8e19 is approximately equal to zero, when you compare it against REALLY big numbers like 1 googol!)

[Steven LaKose]

You may only have a high school education, but you graduated suma comlade from the school of hard knocks...

[Owen Wengerd]

Good catch Julian. I must have misread my slide rule.

[Steve Wells]

Okay, I’m three weeks late reading your article, but I also immediately noticed your misuse of Googol. So, on to entity handles.

Way back in Release 12, if you didn't have the secret password, you could still remove all the entity handles from the drawing. Specifically, in order to destroy handles, you had to:
1. Issue the command: HANDLES
2. Enter DESTROY (no keyword shortcut)
3. Enter the complete string that AutoCAD presented. One of six different strings appeared pseudorandomly. They were:
I AGREE
DESTROY HANDLES
GO AHEAD
MAKE MY DATA
PRETTY PLEASE
UNHANDLE THAT DATABASE

Back then, Autodesk still had a sense of humor. You might recall the brief existence in R13 of some new internal data structures. Just look in the R13c4 ReadMe information under the heading: Resurrecting Zombies. It explained how AutoCAD processed custom objects from applications that had not been loaded. Such objects would reside in the potentially visible AcDbZombieEntity class, and the non-graphical AcDbZombieObject class. If you loaded the app, Zombie Support would resurrect the zombie. The name and idea really were quite appropriate, just as with a traditional zombie; that is, a supernatural corpse reanimated to do the bidding of a powerful master. In R14 the zombie nomenclature got renamed to something a lot less fun and less apt.