Data Mining: Three Steps To Mining Unstructured Data

This article presents some informative perspectives on effective ways to capture and integrate non-static, or "unstructured" data. The term unstructured data is often applied to information regarding an individuals preferences or tastes. This information is viewed as more susceptible to variation and more difficult to predict, yet powerful as a predictor.

One field of non-static data in prospect research may be rating an individuals willingness to give during a campaign. This rating may be tied to a series of factors which often have uncorrelated relationships to each other (affinity for the institution and stock market performance for example) and can change independently. However if they are appropriately aggregated and correctly understood, can be dynamic informers towards predicting the likelihood of giving at all levels.

In our journey of discovery, we have seen one mistake made repeatedly. We have seen static business models and static data models try to be used to model inherently dynamic business processes, particularly at the point of interaction. For example, virtually every customer relationship management system we have come across has a manual classification scheme (or taxonomy) that is meant to be used by the service agent to classify the nature of the customer interaction. This approach has two major flaws.

First, as soon as the classification scheme is published, it is out of date, because interactions with your customers are unpredictable and continually changing. Second, even if the classification scheme was representative of your customer interactions, it is unreasonable to expect any number of service agents to classify their interactions with their customers in a consistent way and with high quality. This very often makes such classification data completely useless, or, more dangerously, misleading. This issue is true throughout the business ecosystem where unstructured information exists.

Read More

Why Mathematical Models Just Don't Add Up

This article presents an interesting perspective on quantitative models of prediction vs qualitative models of prediction. Two main themes can be drawn from this article and applied to prospect research and its utilization of predictive modeling:

1) Be good consumers of research and research techniques. Not every model or technique is a good fit for your questions, or the information available to you (your data).
2) Ask questions outside the box. Instead of just "who is giving" and "how much they might give", ask "why are they giving", and "when might they give" (vs "when might we as an organization ask").

Don't be afraid of "what if" questions either. "What if we managed prospects by affinity rather than capacity, how might our campaign's opportunities for success change?"

Prospect research has barely scratched the surface in respect to analytics, and the opportunities it offers to inform and contribute to our abilities to maximize organizational fundraising potential. Being both critical and creative about what we do as researchers, as well as why we do it, is fundamental to this field reaching new frontiers of success.

Assurances by scientists that the outcome of nature's dynamic processes can be predicted by quantitative mathematical models have created the delusion that we can calculate our way out of our environmental crises. The common use of such models has, in fact, damaged society in a number of ways.

For instance, the 500-year-old cod fishery in the Grand Banks, off Newfoundland, was destroyed by overfishing. That happened in large part because politicians, unable to make painful decisions on their own to reduce fishing and throw thousands of people out of work, shielded themselves behind models of the cod population — models that turned out to be faulty. Now the fish are gone, and so are the jobs.

Read More
Note: viewing the full article requires an online subscription to the Chronicle of Higher Education website.

Can Data Mining and Predictive Modeling Really Make Us Safe?

This article which came out last summer, is particularly relevant given the discussions in recent weeks regarding the Federal Government and data-mining practices. What is data mining useful for? What is it not useful for? These are questions you may ask yourself in your own organizations and projects. There is also a useful link in this article to the comprehensive data mining report produced by the GAO on the governments data mining and predictive modeling projects.

While the scope of your projects may not include National Security concerns, it can be useful to see how others use data mining and predictive modeling techniques to model behavior and forecast future events, from purchasing a house to committing crimes.

In the post-9/11 world, there's much focus on connecting the dots. Many believe data mining is the crystal ball that will enable us to uncover future terrorist plots. But even in the most wildly optimistic projections, data mining isn't tenable for that purpose. We're not trading privacy for security; we're giving up privacy and getting no security in return.

Most people first learned about data mining in November 2002, when news broke about a massive government data mining program called Total Information Awareness. The basic idea was as audacious as it was repellent: suck up as much data as possible about everyone, sift through it with massive computers, and investigate patterns that might indicate terrorist plots.

Read more