The rest of us need to be smarter

Note: This blog is deprecated. @synthesize zach has moved to a new home, at zpasternack.org. This blog entry can be found on the new blog here. Please update your links.

---

This made me simultaneously amused and angered. There’s probably nothing more to be said that hasn’t already been so in comments there and on Reddit. Still, I can’t help but put my $0.02 in.

I don’t know anything about construction, except that it’s hard. To build a house, you need to pour concrete, nail boards together, put up drywall, and a bunch of other things. That’s all hard. If only I had a robot that could do all those things for me... I still couldn’t build a house. Why? Because I also don’t know anything about architecture or structural engineering, or plumbing, or electrical engineering. I’d be lucky to build something that didn’t fall down, and it almost certainly wouldn’t be fit for habitation.

When people say “programming”, they think of the mechanical act of typing stuff into a computer. But that’s really only a tiny fraction of what a programmer does. If you don’t have a reasonable understanding of the innerworkings of computers; if you don’t have the capacity to break a large problem down into ever smaller problems; if you don’t have the ability to visualize every minute detail of a solution; no programming language will enable you to write decent software. That’s all there is to it.

Do you guys remember HyperCard? AppleScript? Prograph CPX? Dare I say it, Visual BASIC? All attempts to make a programming language for non-programmers. And every time one of these hot new “languages for the rest of us” come out, I polish up my résumé. Because once everyone realizes that’s programming is still hard, I get a bunch of new job offers. Best case, all it means is we’re momentarily awash in software written by people who have no business doing so.

That’s not me being elitist. I want to live in a world where everyone has the capacity to create great software, I truly do. I just don’t see it happening in my lifetime.

> those who know it have little interest in simplifying it as it devalues their own knowledge.

Bitch, please. The guy that’s able to engineer a programming language which enables non-programmers to create great software will instantly become ludicrously rich, not to mention ushering in a golden era of information technology. If you believe for one second that some of the smartest people on the planet aren’t working on this right now -- haven’t been working on this for decades -- you are sorely mistaken.

Do you have any musician friends? Go up to one and tell him you tried to play guitar today, and it was hard. Why don’t they make instruments that are easier to play? Let me know how that works out for you.

Any four-year-old of even average intellect has sufficient command of their native language to successfully convey any idea of which they conceive. That no four-year-old has ever won the Pulitzer Prize is proof of the failure of the English language. Clearly what is needed is an easier to use spoken language.

stdout and stderr: Adventures in Redirection, Part 1

Note: This blog is deprecated. @synthesize zach has moved to a new home, at zpasternack.org. This blog entry can be found on the new blog here. Please update your links.

---

For a project I’m working on, I needed to redirect the standard output and standard error of a subprocess I’m launching. While it’s trivial to do this with NSTask (just send it setStandardOutput: and setStandardError:), I had some unique requirements that precluded this. Mainly, I didn’t want the process’ stdout/stderr redirected for its entire lifespan. Rather, I wanted it to redirect only certain parts, and return that data to the GUI through a separate mechanism (a named pipe, actually).

So, what I wanted was to be able to was:
1. Enable/disable the redirection at any point, while the process is running (this more or less precludes freopen- based solutions, as there’s no portable way to restore the streams once they’re reopened).
2. Be able to retrieve and clear the redirected output at any time, regardless of whether redirection is enabled or not.
3. stdout and stderr must both be redirected to the same buffer, and properly interleaved.
4. The helper app (the app that has this code) is written in C++; no Cocoa, no Objective-C.

Based on those requirements, I wrote the following unit test to determine what my interface should look like.

int RedirectorTest()
{
    IOutputRedirector theRedirector;
    std::cout << "This is not redirected" << std::endl;
 
    theRedirector.StartRedirecting();
    std::cout << "This is redirected";
    assert( theRedirector.GetOutput() == "This is redirected" 
        && "cout not redirected!" );
    theRedirector.ClearOutput();
    assert( theRedirector.GetOutput().length() == 0 
        && "redirector not cleared" );

    std::cerr << "Redirected Error";
    assert( theRedirector.GetOutput() == "Redirected Error" 
        && "cerr not redirected" );
    theRedirector.ClearOutput();
    assert( theRedirector.GetOutput().length() == 0 
        && "redirector not cleared" );
 
    theRedirector.StopRedirecting(); 
    std::cout << "This is also not redirected";
    assert( theRedirector.GetOutput().length() == 0
        && "redirection not stopped!" );
 
    theRedirector.StartRedirecting();
    std::cout << "cout";
    std::cerr << "cerr";
    std::cout << "cout again";
    std::string output = theRedirector.GetOutput();
    assert( theRedirector.GetOutput() == "coutcerrcout again"
        && "cout, err not interleaved properly" );

    return 0;
}

Based on that test I wrote the following interface I’d use for my test class:

class IOutputRedirector
{
public:
               IOutputRedirector() {}
    virtual    ~IOutputRedirector(){}
 
    virtual void    StartRedirecting() = 0;
    virtual void    StopRedirecting()  = 0;
 
    virtual std::string GetOutput()    = 0;
    virtual void        ClearOutput()  = 0;
};

I Googled around for various techniques on how to do this, and the most promising one involved calling rdbuf on the cout and cerr streams. In this way, you can specify a different streambuf, and I use that of an ostringstream. It looks a little somethin’ like this:

class CCoutRedirector : public IOutputRedirector
{
public:
            CCoutRedirector();
    virtual ~CCoutRedirector();
 
    virtual void    StartRedirecting();
    virtual void    StopRedirecting();
 
    virtual std::string GetOutput();
    virtual void        ClearOutput();
 
private:
    std::streambuf* m_pOldCoutStreamBuf;
    std::streambuf* m_pOldCerrStreamBuf;
    std::ostringstream m_stream;
};

CCoutRedirector::CCoutRedirector()
: IOutputRedirector()
, m_pOldCoutStreamBuf( NULL )
, m_pOldCerrStreamBuf( NULL )
{
}

CCoutRedirector::~CCoutRedirector()
{
    StopRedirecting();
}
 
void
CCoutRedirector::StartRedirecting()
{
    if( m_pOldCoutStreamBuf != NULL ||
        m_pOldCerrStreamBuf != NULL )
    {
        return;
    }
 
    m_pOldCoutStreamBuf = std::cout.rdbuf();
    m_pOldCerrStreamBuf = std::cerr.rdbuf();
 
    streambuf* newOutbuf = m_stream.rdbuf();
    cout.rdbuf( newOutbuf );
    cerr.rdbuf( newOutbuf );
}

void
CCoutRedirector::StopRedirecting()
{
    if( m_pOldCoutStreamBuf != NULL )
    {
        cout.rdbuf( m_pOldCoutStreamBuf );
        m_pOldCoutStreamBuf = NULL;
    }
 
    if( m_pOldCerrStreamBuf != NULL )
    {
        cerr.rdbuf( m_pOldCerrStreamBuf );
        m_pOldCerrStreamBuf = NULL;
    } 
}

string
CCoutRedirector::GetOutput()
{
    return m_stream.str();
}

void
CCoutRedirector::ClearOutput()
{
    m_stream.str("");
}

It’s pretty self explanatory. StartRedirecting saves off the old cout and cerr’s streambufs, and replaces them with that of our own ostringstream. GetOutput returns the string from that stream; ClearOutput clears it, and finally, StopRedirecting restores the previous streambufs. Easy as pi.

With the test I wrote above this works flawlessly. There’s only one major issue, as I later discovered: while this works for cout and cerr, it does NOT work for stdout and stderr in general. Meaning, calls to printf will not be redirected. I’d not realized that stdout and cout weren’t synonymous. For me this was a deal-breaker, but perhaps others will find it useful.

In the next post I’ll show the technique I ended up using, that works for stdout/stderr as well as cout/cerr.

C Arrays: Here There Be Dragons

Note: This blog is deprecated. @synthesize zach has moved to a new home, at zpasternack.org. This blog entry can be found on the new blog here. Please update your links.

---

I was just answering a question on Stackoverflow (funny how often that inspires me to blog something) about using a C array in a Cocoa app.

The short answer is, you almost never want to do that. Sure, there are times when you might (if you’re going to end up passing it to some C function that needs a raw array; OpenGL or cocos2d, perhaps). In general, though, using some kind of container object is almost always a better solution.

This issue hit home for me, not too long ago. While developing PuzzleTiles, we needed a data model to store game state. Hmm, a two dimensional grid of tiles? My knee-jerk reaction was to type this:

const int MAX_ROWS = 8;
const int MAX_COLUMNS = 8;
@interface GameState : NSObject
{
    // stuff
    tileGrid[MAX_ROWS][MAX_COLUMNS];
    // more stuff
}
@end

Almost as soon as I typed this, I got this terrible sense of dread. As if a million memory buffers cried out, then were suddenly silenced, due to overflow.

But then I came to my senses. I got my big boy pants on, see? I’m an old hand, a seasoned vet, I know damn well what I’m doing. Right? Right.

I made sure to be very careful. tileGrid had no outside accessors; its state was returned via public member functions like - (int) getTileAtX:(int)x y:(int)y Only GameState could muck with the grid, internally, and I was very careful about that. All accessors did sanity checks on all input and output parameters. I wrote code that, in Debug builds, would do an internal consistency check on tileGrid after each tile move operation, to make sure nothing went bad. I knew I had this stuff figured out.

Weeks passed. All memories of tileGrid faded away; we had other issues we were dealing with as we neared our release. GameState was rarely even looked at; it was a pretty simple piece of code, and was (as far as we knew) rock solid. And yet... we were plagued with very rare, seemingly random crashes. Sometimes we’d see them once or twice in a day, sometimes not for several days in a row. If we ever did get to see one in the debugger, or view a crash log, it was complete nonsense. Code that should never crash, was. And it was rarely the same piece of code that failed. It was maddening. For weeks we fought this, scrutinizing every memory allocation and release; running over and over again in Leaks; poring over all our memory management code for any hint of trouble.

In desperation, I eventually returned to GameState. I added some extra consistency checking, and... saw it fail within a minute of running the game. It turned out my original consistency checking code was missing one corner case, and it turned out that case happened pretty often. When it did fail, one part of GameState’s internal code would fail to find a tile, and return the invalid value {-1, -1}. Then another part of the code would not bother to check for the invalid value, and would happily use it to index into tileGrid, accessing tileGrid[-1][-1], effectively randomly overwriting 4 bytes of whatever poor object happened to be 36 bytes before it. Now the random maddening crashes were completely clear.

It wasn’t all bad. While this bug eluded us, we spent a whole lot of time seriously scrutinizing our memory management. In the end, by the time we finally nailed this bug, the entire app was completely bullet proof.

Lesson (re-) learned: if you find yourself using a raw C array, think really hard. Is that really the best way to go? We were careful, and knew what we were doing, and still paid a price. Here there be dragons.