Metrics

Today I learned something -- something that will change how I really approach coding large systems. It's nothing I didn't really know before, but seeing things get into a rightly mess in front of you kind of drives the point home. We had the same type of thing happen at Insurance.com but we never got to the level of bad as I got to see today.When you're looking to build a business of some sort you need to know how to measure how well or poorly you're doing. You can call them Key Performance Indicators (KPIs), or metrics, or whatever it is you want to call them.The key is to treat business like a science. When applying the scientific method you start with a hypothesis, then you design an experiment to test it, experiment and collect the data you hypothesized about, then you sit down and have a think to decide your next move. One of the important things is while you're hypothesizing and designing you're coming up with the parameters that you'll measure and how you'll go about measuring.That's where we stubbed our toe.We have something we're building from a business perspective. We built something and some programmers added some metrics collection where they saw fit.Note that above those are two unconnected sentences.Without designing your experiment properly and collaborating with what to measure you wind up with a hodge-podge of stuff. Worse still, since the data collection are never really part of the overall design they fall between the cracks of testing -- so you have stuff that may or may not be what you need, and on top of that you aren't even confident in what you have.Today I learned by watching the process. It's an obvious lesson that's been learned many times before. But seeing it in front of you lets you internalize it more.