Let the DB Handle It

Perhaps I'm pointing out the obvious, but software developers don't always seem to experience data the same way that, say, a DBA or a BI developer does.

I'm not saying that they don't process in rows and columns, or that they all have Dyscalculia or something.  I'm saying that I frequently detect a reluctance in developers to let the database handle it.  I theorize that this issue can come from a number of possible sources, but is usually a symptom of childhood trauma related to set-based math...

The rationale provided is often one that I consider an artificial barrier in this day and age, which is the old saw that "business logic can't go in the database".  In my decade-plus of experience maintaining, designing and building business software, most of the databases I've encountered have been fairly inextricable from their front-end line-of-business apps from the beginning.  And most of these apps are not going, say, make a switch between platforms or OS's without a wholesale rewrite of the code, anyway.  So using PL/SQL packages or a set of T-SQL stored procs to help encapsulate logic is not a very serious crime.  As long as it's documented, of course.  Also, the best iron in the whole solution always goes to the DB machine. Why not leverage it to accomplish the work of the application? Save some clock and RAM utilization in your web farm and let the database do it.

Another scenario is when the RDBMS is considered a "Mystery Hole," where strange, nigh-impossible things happen in the darkness and hum of the server closet.  I find this understandable enough from developers. I don't see much evidence that good data theory is being imparted on software dev students.  And SQL itself, as a language, operates in a different paradigm from any procedural or OO language, and it requires those nasty set math skills to become accomplished with it.  Advanced SQL techniques do take years to learn, and even to recognize when they can be brought to bear. 
That said, a Pivot operation takes a lot more Java or C# code to implement than it does SQL.  And frankly, being good with SQL means bever having to write a loop. 
It makes sense (to the point of being mandatory, IMHO) for software developers to learn how data works, how SQL and databases work, and when you can leverage them for processing power by letting the database handle it.

A variation on this "Mystery Hole" thing is where it's not about being unfamiliar with the DB environment, but instead it's all about control.  The developer simply can't have any of the logic NOT in code that she has her hands in.  Maybe I'm wrong but, on a big enough project, that way lies madness, methinks.  And again, it fails to heed the wisdom of "rendering unto Caesar that which is Caesar's."  For certain tasks/functions, it's often more expedient and design-time and run time to just let the DB handle it.

Now, I have encountered on many occasions scenes where DBAs and application developers are at odds.  Usually over who gets to do what in whose environment.  In many of these, I wouldn't be surprised to find one of the syndromes above involved.  But there are many times when the obstinacy and regimentation of a DBA will make developers want to punch them dead in the face rather than get involved in DB coding.   So it's certainly a two-way street, and I'm not going to applaud overly-controlling DBAs any more than overly-controlling Developers.  Clearly both parties are on the same team, and should be cooperating to solve the business goals they support, yes?  Rather than playing parochial games over fictitious territories?

Maybe the fact that I'm partial to such compromise (or even averse to conflict) is why I ended up in the BI world.  But as an application developer I always felt it was critical to have insight into the DB layer in order to be a well-rounded architect-level software professional. And if you could get it, working knowledge of the the network/infrastructure layer, too. 

But let's not get ahead of ourselves.  I suppose it's enough to expect that software developers know when to -- well, you know.

Keep Politics From Overwhelming Your Project

Have you found yourself on a project trapped in a client political struggle?  With no dog in the fight except your burning, itching desire to reach code complete and deployment?  Well, you aren't alone.  Consultants often occupy some unique political positions with respect to their clients' organizations. 

Since we are not part of the company, we frequently have outsiders' perspectives.  Over the course of our engagements, we survey how relationships work in these organizations, summarily critique the entire culture, and usually end up just shaking our heads.  Other times we feel just as mired in tar pits as the client stakeholders we work with.  Waiting on legal signatures, approval documents, an internal IT resource to complete a precedent task, etc.  And of course, sometimes we defect because our clients' have better gigs...  ;-)   

Now, if you are one (like me) who specifically tries to avoid the whole subject of internal corporate politics then you must eventually realize, as I did, that in real life not everyone is completely earnest or imbued with a puritan work ethic.  Learning how the political game works for your clients, and knowing who the players are, is imminently valuable even if you don't want to join in.

Because the fact is that our client companies' politics are effectively a force of nature when it comes to running projects.  It behooves us to look objectively at this factor EARLY on in a project -- preferably before getting contracted into an intractable situation.  I have learned through a long chain of missed tells and heartaches to pay close attention to the interpersonal dynamics going on with clients.  Those sorts of things can reveal dangers to be avoided or otherwise defended against throughout the project.  Yell-y bosses, ineffective or power-tripping gatekeepers, impossible-to-please approvers; you can often see it in the very earliest project meetings--if you're paying attention. 

That said, it's frankly quite difficult to track all the various wavelengths of information transmitted by clients in early project meetings: technical context and detail, political context, business context, interpersonal relationship context.  You need at least two people to effectively document/remember/witness it all.  I think it even helps to record audio where possible, too.  As throw-away lines your clients utter can be the missing piece that later keeps the puzzle from being completed. 

No matter how it's done, the point is that you should harvest not only the technical details, but as much of the human context as possible.  And keep those tidbits of info in mind as you proceed.  They can aid you in project planning and in managing the overall relationship on a number of levels:

• allowing for slow-moving companies in your timeline
• selecting which client stakeholders you want to deal with and how often
• identifying your allies and enemies within the organization,
• identifying potential problem decision makers
• finding friends where it's advantageous
• saving you from gaffes that jeopardize your project 

Any of these elements can and will, regardless of your project's technical merit or value, blow it right out of the sky.  Seriously, some person up the line is holding the purse strings for your project, and they can be influenced in many ways that are entirely disconnected from whether you are technically succeeding.  I see very experienced and good consultants spend more time and resources managing these elements than they do on the technical details.  Because in some cases, the political value of the overall consulting relationship is equally if not more valuable than the technical help ostensibly being provided...

Now, if this sounds a little Machiavellian, or perhaps a little "Art of War"-ish, well... it is.  Congratulations: this is consulting. You don't necessarily have to jump in and get your Iago on.  But at least knowing about the game can help you play smart when you have to, and to avoid some potential pitfalls and obstacles to doing your "real" job.

Required General IT Reading

The vast field of IT books has a slew of timeless classics in it, all of which  should be read by anybody intending to be worth their salt as a technical professional.   

But in fact, I seem to encounter less and less discussion/mention of these kinds of books as the blogosphere inflates and the Tweets zip overhead.

So, a few of those in particular I wanted to mention here, as a curmudgeony sort of move:

The Mythical Man-Month (Frederick Brooks) - This is a classic about the human aspects of developing software.  You need this, critically.  Buy it now.

Code Complete (Steve McConnell) - The definitive distillation of programming style and consideration.  For your colleagues, for the future, for God's sake -- read this and absorb it if you ever do any coding.   And don't think you know better.  Because you probably don't.  Sorry.

An Introduction to General Systems Thinking,  The Secrets of Consulting,  Becoming a Technical Leader: An Organic Problem-Solving Approach (Gerald M. Weinberg) - Gerry Weinberg is one of the great unsung heroes in the history of software development.   All his books are chock full'o'wisdom for any kind of systems developer or engineer.  Not to mention his consulting expertise.

The Inmates Are Running the Asylum (Alan Cooper) -  A definitive treatise about bad user interface design in modern technology.  Must-read.  Really, this has saved me some rewrite work over the years.

Design Patterns: Elements of Reusable Object-Oriented Software  (Richard Helm) - A little more down in the details than the rest of these tomes, this one is valuable not just for the specific patterns described, but for illuminating the whole idea that there ARE patterns in the software game...

 So check these out.  All of them definitely affected my thinking and behavior over the course of my career -- in positive ways.  Next time, some more specific BI and Consulting book recs.