Thursday, 2 April 2009

Data mining - digging for gold

Just as coal is the work-horse of modern energy production, so the relational database is the work-horse of modern business. Alright, one is black and dusty and the other is, well, virtual and clean, but the end result is the same - reserves waiting to be mined, whether the reserves are coal or data.

Reserves which may contain gold for their owners.

Before you put on your hard hat with the lamp on the front to break open the server, I’m talking metaphorical gold - metaphorical gold which could be worth a great deal more to your business than the real thing.

First – let’s consider the reserves, which unless you are actually a mining company will be the data stored in databases within your company. Then, let’s look at what the gold might be that’s hidden in the data.

There are few businesses which have not installed a database, whether for managing customers, accounts or stock. As more enterprise-wide systems are installed the amount of data being generated is phenomenal. Some of that data will be immediately accessible through reporting tools. But what could happen if those databases were joined together? What if you could see the sales information together with the customer management information? Or the training data together with sales data? At the risk of mixing enough metaphors to make soup, that would really be cooking with gas ….

But whether it’s one database, or a number joined together, how do you go about looking for gold? Indeed what does gold look like in data terms?

How you find gold is by using a technique called data mining, and what it looks like all depends on your business. It may be customers who are more likely to book a particular type of show in your theatre, or finding which products to bundle together to maximise sales and profit. Or it could be something completely different – depending on what business you are in. The applications for data mining are many and varied and are limited only by business owners' imagination and ambitions.

Data mining is now more accessible and affordable than ever. Products such as Microsoft SQL Server 2008 and 2005 put data mining within the reach of most companies – large or small.

Get in touch if you want to dig for gold in your data. Hard hats with lamps supplied
.

2 comments:

  1. Caroline

    I like the metaphor and imagine that whilst not the same as coal mining it does need skill and expereince to be able to ask the right questions and see the signals that gold might be present. Often that comes from an expert geologist or some one from outside the business who can ask the questions and see patterns that others may be too close to recognise.

    I would be more than happy to share in the gold. The secret is until you look you do not know its there.

    I know your passion for quotes so offer this as maybe relevant in supporting a search, "The difficulty lies, not in the new ideas, but in escaping the old ones." John Maynard Keynes The General theory of Employment, Interst and Money 1936.

    R

    ReplyDelete
  2. Hi Caroline, Absolutely agree the secret wealth of the company that the IT department can help mine is data. We are often too focused on the transactional application that the integrity and maintenance of data is not given sufficient prominence.

    The slight issue with the analogy you use is that a lot of data when mined looks a lot like newly mined coal piled high without any structure. I prefer to think of mining a diamonds and the value comes when it is shaped and cut to show its true worth.

    ReplyDelete