Some great statistics reads...

If you are into analytics or even basic statistics Andrew Gelman is a guy you should be aware of.  He has some similarities to Steven Levitt of Freakenomics fame (both award winning professors under the age of 40) but Gelman is more focused on stats than the self professed math-novice Levitt. Gelman is also a social scientist while Levitt is proudly homo-economicus. They both love to ask and answer questions using data.

Gelman has also produced what is in my mind the best definition of what statistics is: "the study of uncertainty and variation".

Here is a link to Gelman's stats reading list--a pretty broad selection and titles any quant-head should have on their shelves.

As a personal note I would add "Against the Gods - the remarkable story of risk" to this list as the DonorCast selection.

Andrew Gelman on Statistics
Award-winning statistician and political scientist Andrew Gelman says that uncertainty is an important part of life, and recognition of that uncertainty is itself an important step. This is where statistics can help us

Read more

Individual Giving Model--real time philanthropic forecasts!

Analysis of philanthropy and giving trends, as a discipline, has been primarily historical in nature. While researchers have gained a general understanding of the impact of certain economic factors on giving overall (income up = giving goes up), there have been very few "real time" models that can incorporate our shifting economic climate and create accurate predictions and forecasts. Rules of thumb and general trend directions lack precision, and with our mercurial economic climate, lack consistency as well.

Here is some interesting research Center on Wealth and Philanthropy at Boston College. They have designed a model to predict individual household giving in as "real time" as we have ever seen. They call it simply the "Individual Giving Model". They have beta tested"the model on previous years and showed formidable accuracy.

The ability to calculate, within any given year, the impact of economic change on giving can be a wonderful tool in our collective tool box. Please give this paper a review and keep an eye on the IGM.

If the IGM proves successful, the next frontier would be to accurately and timely predict participation, not just total dollars.

Household giving expected to fall
February 17, 2010

When all numbers are in, charitable giving by U.S. households is expected to have fallen by as much as 9 percent in 2009 after adjusting for inflation, a new model predicts.

Individual giving typically correlates to income and wealth, and given the continued challenges Americans face, even the rosiest scenario calls for a drop in donations, says the Individual Giving Model, created by the Center on Wealth and Philanthropy at Boston College.

Assuming slower growth during 2009, the model predicts income will drop at an annual rate of 6.4 percent after adjusting for inflation, and that net worth will grow 4.6 percent.

Read More

Man vs Machine, or "Numbers" vs "Guts"

Through my analysis and recommendations for a variety of clients, I have seen first hand the tension and complex relationship of what I like to call the “pre-analytics world” and the “post-analytics world.” This convergence of two almost fundamentally different perspectives on organizational and campaign planning is still very fresh in the world of fundraising. Analytics represents progress to many in our industry—insights and capabilities based upon a new process of information gathering and analysis. Unfortunately, this evolution (or some might say revolution) has been strained at times.

Many appreciate the technical ability and metrical sophistication gained from analytics and modeling. For some, it is difficult to grasp the concepts used and understand opportunities for application. For others, it is difficult to embrace and trust the insights gained.

Provided with a reasonably well-stocked database, I could offer not only predictions on an institution's future, but also “blind” insights and analysis on what has been happening to-date. Without knowing the information, I could tease out the shift in annual fund messaging strategy, suggest which gift officers were performing well and why, and even reveal strategy for prospecting and solicitation. Impressive? Perhaps. But what happened to good old fashion “gut feelings.”

In the example I present, experience, the strongest factor used in “gut” decision making, is completely absent. I have never spent an hour inside the institution whose profile I could construct. I may offer new insights and perspectives—but don’t really know XYZ University like the VP does. The VP knows the shop and the donors, and feels the campaign is a “go” despite the reservations I might provide.

I can understand why a VP might feel hesitant to plan campaign strategy around analytics work he/she barely understands from someone who doesn’t know the institution as well as he/she does. It’s the institution's campaign, but ultimately his/her job on the line. Beyond campaign success, part of that job is also embracing new ideas and technologies. While he/she may never want to have a fully analytics-driven campaign—rejecting these tools may brand you as a fundraiser from the “20th century,” a wholly undesirable title.

What is the future for “gut decisions” in our world? I truly hope they never go away—and I doubt they ever will. All the modeling in the world could never replace a highly skilled gift officer, or savvy VP. Yet these two groups: pre-analytics (gut and intuition decision-making) and post-analytics (metrics and analytically rooted strategy) are more and more seen as clashing, especially when considering the increased respect and weight given to analytics in fundraising.

What can we do to bridge this divide, and to integrate the best qualities both these approaches have to offer?

This article posits a similar question. While the author does not attempt a thesis-like response, she does offer one sobering and often overlooked factor: “You can't predict emotion with a machine.”

Last week's episode of The Apprentice, filmed at Ogilvy, proved that marketing does not come naturally to everyone. Which is why decades of admen have been held in great esteem for possessing an instinctive ability to produce great campaigns. But, increasingly, the traditional reliance on intuition as the basis for a successful campaign is being surpassed by evidence-based decision making and 'creative experts' should be on their guard.

Read More

Why Demographic Data Just Won't Die

This is a really interesting perspective on what many, myself included, may now consider one of the relic's of predictive modeling: basic demographic data. This data is basic, sometimes clumsy--the data we used in college to learn the techniques of statistics, regression analysis, and econometrics. As analytics junkies today, we all strive to build models and tools to help us fit the contours of the populations we study and to levels much more precise than a zip code or an age group. In modeling, there is “power in numbers,” but there is also an aggregation danger at play when using broad metrics which capture individual behavior and preferences.

I have been posting for some time now on this blog about the frontiers of text-analytics and the raw potential inherent in such custom data mining approaches, that I fear I may have become too nano in my purview.

Behavioral modeling is definitely one of the sharper tools in our toolbox, but read this article and you may find yourself having a similar reaction that I did: reconsidering the benefits and devising new applications for using demographic data.

Demographics: The Targeting Construct That Wouldn't Die

Recently, our customers have communicated a message to us loud and clear. It is a message that may seem counterintuitive here in the 21st century, in the all-digital, micro-targeting, behavioral targeting, contextual targeting age.

Demographics, they tell us, are of paramount importance.

No, seriously. Demographics. Like age, gender, household income. I know; it’s as if I told you I was converting all my MP3s to 8-track, right?

Read More

DELTA Force

Perhaps a misleading, if not corny title. The "spirit" however is relevant to this article.

Thomas Davenport, a respected leader in the field of predictive analytics, spoke at the SPSS Directions conference last month in Orlando, Florida.

DELTA is an acronym Davenport created to capture the life cycle, as well as the environment necessary, for successful predictive analytics ventures. If you have read his book "Competing on Analytics: The New Science of Winning," the concepts will be familiar. If you have not picked up a copy, I strongly suggest you purchase it.

Either way this review of his keynote is informative.

ORLANDO, FLA. -- Walking on stage here yesterday at SPSS's Directions 2007 North American Conference, author Tom Davenport sported a Boston Red Sox cap and used the 2007 World Series Champions as an example of how predictive analytics can give organizations a competitive advantage.

"The Oakland A's had analytics and no money," Davenport said, referring to A's general manager Billy Beane, who introduced the power of mathematics and statistical analysis to the day-to-day operations of running a major league baseball team. "The Yankees had money and no analytics," he added. "The Red Sox have both money and analytics," which he believed contributed to the team's second championship in four years. Not without taking a few additional jabs at Yankees fans in the audience, Davenport, as part of his presentation, "Competing on Analytics: How Fact-Based Decisions and Business Intelligence Drive Performance," proceeded to emphasize the importance of predictive analytics. His formula, he said, could be broken down using the acronym DELTA:

Read More

(Re)-emerging strategies for the “narrative” or “unstructured data” problem.

This article discusses a re-emerging field in predictive analytics called Text Analytics. I say re-emerging, because as the author points out, narrative analysis was a cornerstone of the earliest business intelligence strategies. Today this concept may have utility especially when combined with segmentation or donor-targeting strategies. From prospect management report sheets, phone-a-thon caller logs, to the infamous “other” box on a simple survey question, Text Analytics can provide opportunities for more nuanced insight into the “narrative” data we do have—as well as applications to quantitative models we construct.

One of the fundamental problems of using mathematics to analyze human behavior is the unstructured, or as I like to call it, “narrative” data problem. The amount of purely numerical or quantifiable information available to those in the predictive analytics field is limited—and what this quantifiable information available can tell you is variable as well. I consider non-profit or fundraising analytics to be more opaque than for-profit sectors in respect to this reality. Individuals, on a basic level, need to purchase goods and services. Therefore intent and preference are more transparent. In for-profits, purchasing a product can imply a variety of affinity relationships; this product is a necessity, I prefer this product to other similar products, etc.

Philanthropic giving, monetary or in-kind, is less clear in respect to quantifiable variables producing specific affinity. Attitudes towards institutions or missions may often be more personal than the type of soap you buy, so a donation may imply high affinity. The source of affinity however, can differ greatly: I am an alumnus, my child was a patient, the institution is important to the community, I like the sports teams, etc. Also the absence of immediately available options (there are no supermarkets to choose between charitable organizations) makes comparisons difficult as well. Giving data, capacity rating, alumni classification are all quantifiable values, but some more “narrative” fields like the basic question, “why is giving to us important to you” are more complex.

While the technology for Text Analysis may be more complex and costly than many organizations care to absorb, I believe this represents a very exciting frontier; making predictive modeling more accurate, dynamic, and relevant.

Text analytics is a new IT discipline that has already proved itself in applications ranging from pharmaceutical drug discovery to counter-terrorism to survey analysis, in science, government, and industry. It is poised to break out into the broader analytics market, in workbench form, integrated with business intelligence solutions, embedded in line-of-business applications, and enabling semantic search.

Text analytics is an answer to the “unstructured data” problem, which is best expressed by the truism that eighty percent of enterprise information originates and is locked in “unstructured” form. That problem has been recognized for decades. In fact, the first definition of business intelligence (BI) itself, in an October 1958 IBM Journal article by H.P. Luhn, A Business Intelligence System, describes a system that will:

“…utilize data-processing machines for auto-abstracting and auto-encoding of documents and for creating interest profiles for each of the ‘action points’ in an organization. Both incoming and internally generated documents are automatically abstracted, characterized by a word pattern, and sent automatically to appropriate action points.”

So we see that the earliest BI focus was on text – on extraction, categorization, and classification rather than on numerical data!


Read More

The Next Wave of Business Analytics

Here is a great overview of the current state and future of analytics in the market.

Although Albert Einstein said, "Not everything that counts can be counted and not everything that can be counted counts," organizations in all industries are collecting and storing an increasing amount of data generated by internal transactional systems as well as external content sources. The challenge of what to measure and how to agree on key performance indicators (KPIs) is a point of frustration for both IT and business. However, most organizations are willing to err on the side of caution and deal with more data rather than discarding it.

Read More

The Averaged American - Book Review

The Averaged American: Surveys, Citizens, and the Making of a Mass Public by Sarah E. Igo traces the history of market and survey research. BusinessWeek provides this solid book review. I have not yet read this book, but it is on my short list.

Polling, once considered a scandalous invasion of privacy, is now an accepted practice. More than 20% of Americans were polled at least once in the past year. As Igo aptly concludes, "we will continue to live in a world shaped by, and perceived through, survey data."

Read More