Data mining - not just for the "good guys"

Data Mining is the use of intelligent software tools to look for interesting patterns in large volumes of data. Credit card companies do it - to look for unusual patterns of spending to try to prevent fraud for example. Financial institutions do it to try to work out what they can "cross sell" to their customers, or to determine who is credit worthy. Law enforcement agencies use it to help trace murderers

With that in mind, I've been meaning to blog this for some time. It happened in America but could easily happen in the UK to any large repository of personal data. Records of Armed Forces Veterans and active personnel have been stolen. More recently 1,500 employees of the National Nuclear Safety Administration have had their details accessed by hackers unknown. The article raises the spectre that the data is sufficiently extensive to be "mined".

With volumes of data this large, it's not just the danger of individuals being impersonated that is the concern. There is the emerging problem that data mining could be used to search for important patterns that could be used by [insert bogeyman of choice here].

The technology to intelligently mine data is only a free download away. The desire to "get at" data, whether from curiosity or some more sinister motive is well documented. The central storage of volumes of personal data in the National Identity Register will be one huge weak point that will be hard to protect.