Monday, April 23, 2007

Objects, just not Object Oriented

I've been on a real programming language kick for the past year. First, I picked up Prolog again. Then Javascript, then Ruby, then Lua, then Erlang. This is in addition to lots of programming in C#, some in Java, and a little C++ on the side. I'm starting to notice lots of similarities between languages. I don't mean superficial similarities (Java has classes. Whoa, so does C#. Dude...). I mean deeper, conceptual similarities.

As an example, take objects. C++ has objects. Smalltalk does too. Also Java and C# and a whole lot of other languages. However, all of the objects in those languages are bound together by sharing a single class.

Javascript and Lua don't share that requirement. They're still object-based (you have entities which hide implementation and expose functionality), but object's don't necessarily share a common class. In fact, an object in these languages can (and often does) have no class at all.

Further still, some languages have object-like things. Erlang, for example, allows you to create processes. Processes can send messages to each other. Processes can even have private data (since a process is just a function, and functions can always have local variables, processes can have private data). Is this an object? I think so!

Some languages even give you object-like functionality in even stranger ways. Take purely functional languages like Lisp (or Scheme) or ML. These languages let you create "closures". A closure is a function that exists within (is closed to) an environment. In Javascript (not a functional language, but one which supports closures):

function make_add(left_addend) {
var f = function(right_addend) {
return left_addend + right_addend;
}

//At this point, f is bound to left_addend. Whatever
//we passed in as left_addend is now known by f.
//We can return f, and it will correctly remember
//the value of left_addend.

return f;
}

var add10 = make_add(10);
var add1000 = make_add(1000);

print_to_console(add10(5));
print_to_console(add10(50));

print_to_console(add1000(5));
print_to_console(add1000(50));




15
60
1005
1050


Why is this cool? Well, we have effectively created an entity which has private data. We could go a step further and make 'f' into some sort of dispatch method. It could take, as its first parameter, an operation. So, perhaps we could have this:

function make_number(value) {
var f = function(operation) {
if (operation == "add") {
return make_number(value + arguments[1]);
} else if (operation == "sub") {
return make_number(value - arguments[1]);
} else if (operation == "toString") {
return value;
} else {
return null;
}
}

var number10 = make_number(10);
var number1000 = make_number(1000);

print_to_console(number10("add", 5)("toString"));
print_to_console(number10("sub", 50)("toString"));

print_to_console(number1000("sub", 5)("toString"));
print_to_console(number1000("add", 50)("toString"));




15
-40
995
1050


Not too pretty, but it should work. Object-oriented programming in functional languages. Who knew?

Big, Bigger, Biggest Design Up Front

This was originally posted at Ternary Software. It is mirrored here because I wrote it and I want it on my personal blog as well as my work blog.



Background

Extreme Programming encourages its practitioners to do away with a technique known as Big Design Up Front. BDUF is a software development process that has existed for nearly as long as software development. The general idea is to design a software system completely on paper before writing a line of code (experiments and proof-of-concepts aside). XP claims that this is a bad idea. In my recollection, the XP book claims that you wouldn’t get the design right anyway. You really can’t get the design right until you have finished writing the code.

Reality

In theory, this doesn’t sound too bad. I mean, the best way to understand a system is to try to formalize it in code. You’ll spot all of the inconsistencies and problems pretty quickly. (In fact, I think that’s the real skill that a software developer brings to the table - the ability to analyze a system and find the problems). But still, to do no design up front? Clients don’t come to you with no design. In fact, if one did, would you really take their business?

Prospective Client: Boy, do I have a project for you!
Development Team: Great, what do you have in mind?
Prospective Client: Well, see, it’s a thing. It does stuff. It’s a lot like That Web Site, only different. By different, I mean really different. But I’m not quite sure how. Also, it transforms into a boat.
Development Team: What? That makes no sense!
No Longer Prospective Client: Well, I thought I’d design it as I go.
Development Team: Well, we’re really booked right now, but we’ll call you as soon as we have some free time.

So I suspect that developers, too, need to do some design up front. I can think cases where NOT designing up front has cost us a lot of effort in re-design. Developing a 2-tier system when we knew that the client really wanted a 3-tier system is but one example. (We fixed that problem within a month, but it was still a hassle that could have been avoided). But how much design is too much? The XP book says something about doing “enough” design, but that doesn’t really provide a lot of guidance.

Disclaimer

Now we get into the part that I don’t really understand well enough. These are my thoughts, but they’re surely wrong. My thoughts will probably change with time, and I would appreciate any feedback that I can get.

3 Kinds

Here’s an over-generalization: systems that you design can fall into 3 categories:

  1. Systems that perfectly implement the requirements of today with no thought to the future. Any attempt to use them in a way that is different from the original intent will require massive amounts of redesign. I’ve seen a lot of Javascript code that seems to fall into this category. For that matter, a lot of code written on consulting projects seems to fall into this category.
  2. Systems that implement any conceivable requirement for the next 5 years. Sometimes, they do a good job of this; sometimes, they fail miserably. Yes, ZZZManager, I’m talking about you! A lot of libraries tend to lean in this direction.
  3. Systems that, while implementing the requirements of today, also have a lot of hooks for the requirements of tomorrow. The assumptions that they make (and the hooks that they provide) aren’t necessarily perfect, but it’s not TOO hard to change them to support new requirements. Some really good libraries and really good frameworks do this. When I saw how simple it was to write a Java Servlet, I remember thinking that Servlets strike an excellent balance of providing functionality to the programmer without making too many assumptions.

I would furthermore attempt to characterize those three kinds of designs as:

  1. Not Enough Design
  2. Too Much Design
  3. A design that’s juuuuust right.

The Source of All Design that is Good

Let’s assume that I’m correct in my characterization. How do you get this kind of design? Well, I think it’s a little like Chess. For chess masters, the match is won by the person who can “see” further into the future. I think that good software design comes from people who can (correctly) guess the future. There are a number of contributing factors:

  • You yourself. I think some people just naturally think more about the future than others. I don’t think that people who think about the future are better at designing (see Too Much Design). But I do think that it’s important to be thinking about both the present and the likely future.
  • Your experience. If you’ve seen a problem before, there’s a chance that you will try to avoid it in the future. I think that experience can help you avoid problems without even realizing that you are avoiding them - you just make really good choices.
  • The pressures on your team. I think teams that have some healthy amount of slack time will be more likely to think about things that are a little broader than their current task.
  • Your level of attunement. The more in-tune you are with your client and their needs, the more likely you are to guess correctly about their future requirement needs. You need to understand not only their product, but to a certain extent, their business model and their users.

Tips

Those factors are often hard to change. Are there more immediate guidelines for avoiding problems? I certainly don’t know how to avoid all problems, but here are some general thoughts that have helped me so far:

  • If, while implementing a feature, you think you will be ripping out the code in a month, you probably need to design a little more. Changing code in a month is OK, but replacing it is scary. Replacing it in 3 years is probably OK.
  • If you can’t explain to the client what you are working on and why they need it, you’ve probably designed the wrong thing (i.e. over-design)
  • Don’t take shortcuts. If there’s an easy way and a hard way to design a system, make sure that the easy way isn’t really the “really hard later” way. It might be perfectly fine to take the easy approach. In fact, you probably should. Just make sure that you’re not going to get screwed in the near future.
  • If you are writing very similar (but also slightly different) code in several places, you probably need more design. For example, you use an Excel writing library in 20 places in the code base, something is screaming to be extracted into its own class (or set of classes).
  • If your code has no layers, you probably need to design some more. Just as classes need to be small and simple, so too must groups of classes. If you have a class that is referenced by code in several namespaces, you should wonder why everybody needs to know about him. (Perhaps everyone really does need to know, but you should still think about it).
  • If you end up writing a lot of static methods, you need to extract some classes. A few static methods (especially named constructors) are OK, but if you have too many, you’re just writing procedural code.
  • If you end up writing a lot of private methods, you need to extract some classes. A few private methods are OK, but if you have too many, you’re just writing procedural code.
  • If you know that some of the code that you are writing is more general than your specific task, break it into 2 pieces. Write some of the code in a general way (using Template Method, aggregation, or something cooler), and write your specific code in terms of the general pattern. LISPers (and other functional programming dudes) know how to take this to the extreme. (Every functional language that I’ve encountered comes with a built-in function that can apply a function to every element of a list. Finding the sum of a set of numbers is often as simple as applying “+” to every element of a list). Be very wary of this, however, as this can lead to some scary over-design.
  • If you don’t know what you are going to be doing later this week, you can’t anticipate the future. Have a plan.
  • If your plan describes what you will be doing for 6 months, you will be using a different plan in 6 months. You’ll probably write code that won’t make sense in that new plan. Spend less time designing, and focus on short, iterative releases.
  • If you write code that everybody uses, and which affects code throughout the system, and your code needs to be changed whenever some other requirement changes, you have too much centralization. You’ve designed a fancy system, but it’s going to cause you problems.

Conclusion

Those are all of my ideas for now. Actually, I could keep going, but I would just be rambling more. I want to again stress that this area is very unclear to me, and what I have typed here are only my current thoughts on the subject. I would love to hear other people’s opinions!

On Windows CE, Microsoft, the framebuffer, backwards compatibility, and demons

This was originally posted at Ternary Software. It is mirrored here because I wrote it and I want it on my personal blog as well as my work blog.



I spent the better part of today struggling to get some fancy graphics on my O2 XDA Exec (read a review if you are interested in the device). I know that I can use the Windows CE GDI to draw the graphics on the screen. I also know that will be slow. Not necessarily too slow, but certainly not an API that any self-respecting game developer would use. So, moving along, I knew that there was the GAPI - the Game API. It sounded great - provide access to the framebuffer and to low-level keyboard functions. So I read and implemented.

Everything was working fine in the Pocket PC emulator. But on a real device, no luck. It ran, but incorrectly. For one, it pretended that my device had a resolution of 240×320 pixels, which is 1/4 of the actual number. It used pixel doubling to fill the whole screen. On top of that, it wouldn’t draw anything in the leftmost pixel column. I had an ugly blue column extending from the top of the screen to the bottom.

I spend hours looking for a solution. I looked to see if it was a clipping problem, an API misunderstanding, a device issue. I couldn’t find any information. Then, I started reading support forums for other applications. It was there that I discovered that the problem is that, at this point, GAPI is provided on new devices as an emulation library. Apparently, in the Pocket PC 2002 / Windows Mobile 2003 days, GAPI was the way to make games. However, a lot of game developers made a lot of assumptions about transient things like, say, the screen resolution of devices. Microsoft, fearing that high resolution screens would break a lot of existing applications, decided to release a GAPI implementation for more recent devices that doesn’t return a pointer to the ACTUAL framebuffer. Instead, the application gets a pointer to a back buffer. Upon calling GXEndDraw(), a GAPI function whose purpose should be reasonably obvious (it flags that drawing operations are complete), this emulation library flipped the buffers and (on my device), upscaled all of the pixels.

HOWEVER, certain crafty game developers apparently thought that GXEndDraw was a useless function. After all, it didn’t seem to do anything on their Windows Mobile 2003 devices. And, if it doesn’t do anything, why should they take the time to call it? I believe that this is why I was having trouble with Pocket Quake. I plan to try to fix it and then tell the developer.

As for the single unwritable column of pixels, I simply think that this is a bug in the copy of gx.dll (the GAPI runtime library) that is on my device. The pixel doubling routine is probably wrong. I believe this because I think I have solved my problem. I can write all 480×640 pixels, including that annoying leftmost column.

To perhaps save somebody else the time that I just spent, let me share my findings. Microsoft suggests using the GETRAWFRAMEBUFFER escape operation in the GAPI Legacy Support section of “Developing DPI-Aware Applications”. There’s a good example in that article. This seems to be the new way to really, definitely, for sure this time, get raw access to the framebuffer. Until some developer takes too many liberties with it, and Microsoft has to invent yet another hack to keep other people’s applications working.

I should probably just bite the bullet and use DirectDraw. I thought that I had read on the GAPI for Handheld PC’s page that DirectDraw was not supported on the PocketPC platform, but they were perhaps referring to the 2002/2003 platform. I don’t see DirectX dlls on my device, but perhaps they are hiding.

Perhaps I’ve read so many of Raymond Chen’s posts that I’ve started to sound a little like him.