Friday, April 29, 2011

Hippie's Backyard Redux: Data vs Information

What's the difference between data and information?  
It's not a trick question.  Data are (or "is", depending on your grammarian tendencies) unadorned facts,  while information consists of facts evaluated and/or interpreted to provide a particular meaning. It's the difference between plotted points on a graph, and a trend-line drawn to connect them.  The points are interesting, but the line tells a story.  Data points have value, but to really get answers you want information. Therefore in order to answer business questions, a typical reporting user needs and wants plenty of information at their disposal, all based on dependable data. 

So then why is it that these users typically end up with a sea of data points, but are left on their own to derive any usable information?

Tough one...

I know there are some organizations that simply don't have the processes or mandate to find and fix inefficient processes -- including reporting processes.  The scope and depth of a problem process simply sits in an organizational "blind spot".  Northridge has been called in to "fix" some serious manual-extract, chicken-wire-and-duct-tape Excel situations, fed through a shadowy underworld of data connections hidden to the non-adept, where if users can get any kind of realistic data at all, they're thankful.  Of course, if that kind of solution is already causing pain, you're too late for duct tape.  But somehow the company can't see the larger-scale syndrome at work.

Also I've seen more than one "bottom-up" reporting system, where the shape and structure of the user-facing solution is based largely on the data and the structures it occupies, with little or no recognition of business-based reporting needs.  In these situations, the user frequently must sift through arbitrarily-named (from their perspective) metrics, segmented in senseless ways (again, from their perspective) in order to get answers.  Or they get data access along lines skew to the business roles they fill, restricting them along axes of data volume or visibility.  "Data, data everywhere, nor any conclusions to be derived."  Of course, by then time this has been built, there's neither budget nor guts to actually build a bridge to the users.

And of course you can never underestimate the sheer inertial power of "We've always shown it that way"  ...

But in all cases, let's face it -- the goal is to get OUT of that situation where you're giving the right stuff in the wrong format, and the right people can't find it anyway.  So, the key manuever is to evaluate one's reporting offerings through the lens of business context. Context, which sums up in one word the difference between data and information, gives users the background they need to derive conclusions, make decisions, and ultimately take action when they look at a report.   In practical terms, you're showing users graphs with trend lines instead of grids of numbers.  Using indicator graphics to show whether costs are up for current period over previous period.  It also means organizing your reporting data in a way ammenable reporting needs.

Detailed data will still have its place since certain types of analysis may require a deeper look into the data behind an informational trend.  But that's why the good lord invented drill-down logic.  I refer to it as a "tip of the spear" strategy.  You want to start with a summary, and allow further exploration along established data relationships.  Click the "up vs down over prior period" indicator and it shows a grid-style report with daily details.

Accounting for users workflow-oriented interactions with data can help establish a more coherent and useful representation of data.  This structuring of information based on users' needs is called Information Architecture, which is defined by the Information Architecture Institute as "The art and science of organizing and labeling web sites, intranets, online communities, and software to support findability and usability."  And it can mean the difference between efficient and effective information delivery, and a hippie's backyard.

Friday, April 15, 2011

Keep Politics From Overwhelming Your Project

Have you found yourself on a project trapped in a client political struggle?  With no dog in the fight except your burning, itching desire to reach code complete and deployment?  Well, you aren't alone.  Consultants often occupy some unique political positions with respect to their clients' organizations. 

Since we are not part of the company, we frequently have outsiders' perspectives.  Over the course of our engagements, we survey how relationships work in these organizations, summarily critique the entire culture, and usually end up just shaking our heads.  Other times we feel just as mired in tar pits as the client stakeholders we work with.  Waiting on legal signatures, approval documents, an internal IT resource to complete a precedent task, etc.  And of course, sometimes we defect because our clients' have better gigs...  ;-)   

Now, if you are one (like me) who specifically tries to avoid the whole subject of internal corporate politics then you must eventually realize, as I did, that in real life not everyone is completely earnest or imbued with a puritan work ethic.  Learning how the political game works for your clients, and knowing who the players are, is imminently valuable even if you don't want to join in.

Because the fact is that our client companies' politics are effectively a force of nature when it comes to running projects.  It behooves us to look objectively at this factor EARLY on in a project -- preferably before getting contracted into an intractable situation.  I have learned through a long chain of missed tells and heartaches to pay close attention to the interpersonal dynamics going on with clients.  Those sorts of things can reveal dangers to be avoided or otherwise defended against throughout the project.  Yell-y bosses, ineffective or power-tripping gatekeepers, impossible-to-please approvers; you can often see it in the very earliest project meetings--if you're paying attention. 

That said, it's frankly quite difficult to track all the various wavelengths of information transmitted by clients in early project meetings: technical context and detail, political context, business context, interpersonal relationship context.  You need at least two people to effectively document/remember/witness it all.  I think it even helps to record audio where possible, too.  As throw-away lines your clients utter can be the missing piece that later keeps the puzzle from being completed. 

No matter how it's done, the point is that you should harvest not only the technical details, but as much of the human context as possible.  And keep those tidbits of info in mind as you proceed.  They can aid you in project planning and in managing the overall relationship on a number of levels:
  • allowing for slow-moving companies in your timeline
  • selecting which client stakeholders you want to deal with and how often
  • identifying your allies and enemies within the organization,
  • identifying potential problem decision makers
  • finding friends where it's advantageous
  • saving you from gaffes that jeopardize your project 
Any of these elements can and will, regardless of your project's technical merit or value, blow it right out of the sky.  Seriously, some person up the line is holding the purse strings for your project, and they can be influenced in many ways that are entirely disconnected from whether you are technically succeeding.  I see very experienced and good consultants spend more time and resources managing these elements than they do on the technical details.  Because in some cases, the political value of the overall consulting relationship is equally if not more valuable than the technical help ostensibly being provided...

Now, if this sounds a little Machiavellian, or perhaps a little "Art of War"-ish, well... it is.  Congratulations: this is consulting. You don't necessarily have to jump in and get your Iago on.  But at least knowing about the game can help you play smart when you have to, and to avoid some potential pitfalls and obstacles to doing your "real" job.

Friday, April 1, 2011

Reporting: The "Hippie's Backyard" of Software

As your Business Intelligence consultant, acting in my capacity as a reporting architect, I'm afraid I have to inform you that your current Reporting "system" is what we call a hippie's backyard.

Just think about it a minute.  Yes, it's a technical term.

Not important now.   The important thing is, it's not working for you or your organization. At all. In fact, I'd be inclined to describe the situation as reversed; you're working for it.  After all, how many man-hours do you waste annually on shuffling data in and out of Excel from one internal silo to another?  What kind of a report is a 26-tab spreadsheet with 64,000 rows of 50 columns of 8pt font in each tab?  And how many differing copies of those spreadsheets are currently strewn across network and thumb drives?  How many are the same?   Why didn't the developers of your Inventory system include any basic reporting functionality?

I'm just asking.  You don't have to get defensive.

Listen, I know how it goes: At first, there was supposed to be a set of reports included in the initial release.  But then the timeline got stretched, and you ended up with no real process, no real tool to get anybody any reports, just totally ad-hoc data requests fielded by whoever can do it whenever they can.   So then Ted in Accounting sets up an Excel thing that ties to the application database, people start using it, and data grows. Versions propagate.  Probably 6 or so around the network, some of them like 3 years old.  Two years pass, more users get on board, and the "Financial Data Mart" (as it's become known) has maxed out on file size, crashes itself at least once a session, crashes the Inventory application daily, and runs like cold molasses.  But there's no time to do anything about that.  Besides, Ted quit 6 months ago.  Who's going to do it?

Well, isn't that really why you called me?

No, no -- you shouldn't feel bad.  The same phenomenon is EVERYWHERE.  Reporting is the great red-headed stepchild of the IT world.  Ask any true-blue hacker how much they like building reports...  But here's the way to think of this:

At a certain level of organizational size and complexity (which will be different based on industry, culture, and other factors) a simple ad-hoc reporting system is insufficient.  You can't run it with a single person fielding 20-30 slightly different requests for the same data.  That setup doesn't scale, and can't handle the request traffic, can't handle the data volume, can't take the heat.  And truthfully, the organization will waste money trying to resolve the issue by throwing labor at it. The solution is to consider the problem holistically and strategically, and to provide a robust, organization-wide reporting solution.   The end goal is to establish a solution that provides all workers the information they need to do their jobs -- on-demand. 

How?  Well, first things first.  We need to do a broad inventory of prioritized reporting needs -- who needs to see what when, and what format works best for them.  That inventory is analyzed to generate a global information architecture, which organizes the various categories of information based on user need and natural hierarchies within the organization.  Next, the Data Structures at work in the organization must be analyzed to determine their usefulness vis a vis the information architecture, their accessibility, and their availability. When we bring the results of this analysis to bear on the Information Architecture, we can map specific information needs directly to data sources, we can design a truly centralized reporting solution. 

And, yes, we get your company out of the hippie's backyard.

So, now let's talk a little about just how many report requests are being made, by whom, and what information is in them.  And then we can talk about where all of that data comes from...