Archive of published articles on September, 2010

Back home

All About Documentation


There is a recent article on Gamasutra: Game Dev Collaboration: Google Docs Style. Every developer has his own story about the failures of documentation. I would have hoped that those thinking about this the most have already moved past the ‘silver bullet’ naiveté Chris shows in that article (well, what do you expect from producers…), but apparently not. Which is sad, because the problem is a single concept with really only two solutions.

The problem is usability.

The solutions are resignation or intense investment.

No one wants to document because 1) It is a pain to document using any software or system, including Google Docs, and 2) It is even more painful to update documentation using any software or system, including Google Docs. When a virtual process is painful, there are always things we can do to make it less painful (and ideally, painless). Blah blah blah. The problem is obvious. The solutions are not.

Solution 1: Resignation. Don’t even attempt to document.

Obviously that is a bit of a generalization. Really, it means don’t force, standardize, or expect documentation. People will document things on their own, using the best ways available, when it becomes an obvious benefit. Ultimately, people are just asking each other for information, and remember that developers, like all humans, are incredibly lazy. How many times have you not bookmarked a document link someone sent you? How many times have you kept open an IM window because you don’t want to write the info down? This is natural. We are usually asking for help because we’re doing something unusual, outside of our day-to-day, and the truth is we don’t need this info that often. Documentation is not that useful when someone is learning something for the first time. It is more effective to learn from someone and the benefit of documentation is slight, especially when the documentation is not kept up to date.

All the wiki-cloud-collaborative-meta-web-editor software in the world isn’t going to change these facts. There is a barrier that even Google Docs, which I think is the easiest-to-use software for documentation available, cannot itself overcome.

So, like I was saying, doing nothing is probably fine, even for large projects. On small projects, communication is better. On large projects, things move much faster than the few foolhardy souls who are willing to document can keep up with. Things will get documented occasionally, and everyone will expect the inevitable, and who knows, maybe we’ll build our schedules to take this into account and we’ll build more intuitive systems, tools, and interfaces.

The one exception to this is for outsourcing, which I’ll touch on later, because the logic for an internal development team is inverted with outsourcers.

Solutions 2: Intense investment. Smart people make a serious effort.

The problem with documentation is that the systems for managing it haven’t achieved a high enough level of usability. The cause of that problem is, our best and brightest haven’t really worked on the problem. It doesn’t get respect. Which is strange, when you consider what we have figured out. We in games deal with a product that thrives on and is reliant on usability, and no one I know of, even those studios with awesome tools, have figured out documentation.

I don’t have an actual solution. I am working on one, and hopefully can implement it at BioWare, but it is just one possible solution and who knows if it’ll work. My point is that we don’t take it seriously. We don’t even think about it correctly. We treat it as a separate entity, but that treatment has failed us. It is a subset of tool design, the same way UI and menu design is. If you haven’t thought about documentation as a living component of your tool, your tool has poor usability, no matter how actually usable your application is. This ‘integrated documentation’ thinking is common in shrink-wrapped applications; just think of context-sensitive ‘F1′ that brings up help to the current word under the cursor in a code editor, or activity or menu generally. The added complexity in our industry comes with the fact that our tools and systems are constantly evolving and changing, and docs are almost always written and updated in spare unscheduled time by development team members- two complexities which generally do not exist for shrink-wrapped applications.

We need to start treating documentation as a first-class issue and component of our tools, no matter how bad our tools are. That doesn’t mean scheduling time, it doesn’t mean creating a documentation superstructure that sits on top of our actual tools, like wikis/google docs/word docs do. It means really, truly integrating documentation into design and implementation of tools, and having your best people work on the tech for it, and having all the documentation happen by the actual users (so hard-coded message boxes that display info are out).

Hopefully in a few weeks I should have some progress to show on my collaborative documentation framework, to back up what I’m talking about.

Addendum 1: Outsourcers
If you are deploying systems to outsourcers, there need to be changes to the above. Outsourcers should not be given tools in such a state of flux, period. If you can’t lock something down, you are doing outsourcing wrong, or you shouldn’t be outsourcing it. Once it is locked down, go ahead and document it, and it is the job of the developers who manage the outsourcers to keep the documentation up to date and to roll out changes.

Addendum 2: Wikis
Hailed years back as the answer to documentation problems, wikis haven’t worked (and I’m sure we’ll see the same with Google Docs if we continue the current trend). The reason is primarily (IMO) that Wikis use a system of organization completely different than a Word document. It is non-hierarchical, non-procedural, and has no organization. It makes a Wiki wonderfully suited to describe things (it works great as an encyclopedia), but terrible at documenting processes. A proper wiki has lots of cross references and modular information. How useful is this to document workflows? I’d say it is counterproductive, as I know I’ve been enticed to spend an entire afternoon adding dozens of pages to a wiki that ultimately actually give too much information for the user and are hard to follow. So all you’re left with on most wikis is a bunch of pages that basically read like Word docs anyway (all slash commands in the game, what the different fields of Tool X mean), with a shittier text editor than Word and equally inconvenient way of accessing information. Which is why I’d prefer google docs- it does away with all the pretentious wiki bullshit and just gives you a bunch of actual documents, that are easier to collaborate on (in real time!).


What’s wrong with Autodesk? Part I, Feedback Mechanisms


In the past month, I was part of a conference call with Autodesk and industry technical artists about integrating .NET into Maya (and how we use it in game development), and a couple weeks later we also had a couple Autodesk reps visit BioWare Austin to talk about their consulting services and get feedback. Both were very honest and constructive discussions. I said what I wanted to say and felt I got honest answers. It was fun to take them to task about a variety of things, but more importantly, it gave me a much more nuanced understanding of the external problems with the company. I’m going to go over them in the next series of blog posts.

Problem 1: Feedback Mechanisms

Once, when a woman made a request of him as he passed by on a journey, he at first said to her, “I haven’t time,” but afterwards, when she cried out, “Cease, then, being emperor,” he turned about and granted her a hearing. -Cassius Dio, Roman Histories, 69.6.3

The above is a famous quote about the Emperor Hadrian, who was famously approachable. It is this lack of approachability that is the first and most fundamental correctable problem with Autodesk. There is simply no good feedback mechanism for Autodesk users. (I’ll explain in my next post how this is technically false but effectively true). Whenever I rip on AD to their faces, I always have brief guilt when they respond with ‘You need to tell us about it so we can fix it.’ And they are, of course, correct.

I have the same issue with my tools at work- if people are having issues, I need them to tell me about them. Every day. Until I do something, even if that only means getting it into Hansoft and on the schedule. But they don’t- the teams that do it the best have been cultured to do it. The animation team, which I’ve worked with the longest and thus has the longest history of support, is the best at requesting tools and demanding support. The environment team, which has traditionally had little and poor quality support (though, admittedly, far simpler requirements) often doesn’t respond even when things are crashing.

I think we, as AD customers, suffer like my environment team does. We are not used to getting support from AD and are unsure of the people we’re dealing with. This falls squarely on AD’s shoulders; when we report issues, we need acknowledgment even if we can’t get immediate results. When we see changes, we need to know what prompted them, and we need to know when our prompting causes changes.

In most ways, my position and Autodesk’s is fundamentally the same- we are both tools producers. The difference is the route feedback and requests take. In my case, they all must be balanced against a schedule driving towards a creative vision (a videogame, film, whatever). The people using the tool must communicate upwards to the people scheduling me, and they schedule me to work on what they deem most important. There’s conflicting pressure from below (users) and above (planners). In AD’s case, there is not that conflicting pressure. It is the users that should be driving the development of the middleware. It is still important for AD to provide a vision (and I don’t believe they have the right people to do this right now), such as ‘a plugin model architecture’ or ‘bloated with features’, but that vision is only relevant towards how it accomplishes customer requirements (‘creating a clean API’ or ‘writing a closed but fast tool’).

This lack of feedback mechanism is undoubtedly the biggest problem I have with Autodesk- they are a company that doesn’t seem to communicate with their users. But there’s actually a dirty little secret only the most privileged of lead artists at the most wealthy of studios know: Autodesk actually has important and serious feedback mechanisms. Confused? You should be.