Platt Perspective on Business and Technology

Connecting into the crowd as a source of insight and market advantage – 6: collecting the right data and analyzing it -1

Posted in macroeconomics, strategy and planning by Timothy Platt on June 17, 2012

This is my sixth installment in a series on connecting into the crowd as a source of insight and market advantage, but with an important difference – doing this in ways that explicitly allow and support measurement of costs and of value returned in the interactive online context (see Macroeconomics and Business, postings 77 and loosely scattered following for Parts 1-5.) I have been discussing Web 2.0 and social media, and the increasingly globally ubiquitous online conversation that every business now faces, in this series. And I have been discussing how they are both bringing outside and crowd sourced voices into a business’ marketing and shaping that business’ own needs and priorities as it reaches out with its own business-sourced message.

At the end of Part 5: adding in a fuller range of online interaction and crowd participation 2 I stated that I would discuss data analysis and statistical data analysis in particular in this installment, as an approach for developing value from data and insight obtained in tracking and measuring this flow of information. And I want to start that by making a crucially important observation:

• From an operational and a practical hands-on perspective this should have been the topic of my first series installment, and for a very simple and important reason. Effective, meaningful statistical analysis of data obtained and developed has to begin before the first data point is collected. You can only perform meaningful statistical analyses that you can gain action-enabling insight from, if you collect data that is by its nature is susceptible to meaningful statistical analysis.

Anecdotal evidence cannot be grouped and organized through statistical analytical processes to develop models that show performance patterns. Such evidence cannot tell you systematically how your business has reached the operational and marketplace position it is in now and it cannot tell you precisely where you are now. And anecdotal evidence in and of itself cannot in general provide the insight needed when planning and carrying out next steps for a business.

• What in this sense is anecdotal evidence? It is free-form, unstructured data that cannot be systematically ordered or grouped for comparative analysis across cumulative data sets.

Most statistically analyzable data is either numerical in nature, or it assumes yes or no or similar Boolean values. Or it can be organized in a manner through which it can be consistently, replicably represented as numerical or Boolean data. I will discuss certain exceptions to that in the course of this part of this series, and particularly with regard to tools such as cluster analysis and its particular role in marketing.

• But even numerical data is not always subject to clear-cut statistical analysis. Data sets have to meet specific requirements if they are to be applicable to such analytical processes and tools.
• First and foremost you need to have enough data in order to be able to perform meaningful statistical tests at all. Different statistical tests require different minimal data set sizes but they all require at least minimal quantities of reliable data if they are to give you meaningful results.
• And the more complex the questions you would seek to address through statistical data analysis, the more data you will need at minimum. In practice, “more complex business questions” means more variables that you would have to analyze. Consider as a working example in explanation of that, asking more complex and nuanced questions involving more factors [variables] related to predicting the behavior of individual customers who fit into specific demographic profiles, or individual transaction processes that they enter into. The more complex your questions the more complex the data that you will have to have available to statistically address them and the more data you will need at minimum.
• And the smaller your data samples above the minimum necessary, the greater uncertainty in your findings – larger data sets translate directly into greater certainty that statistical models developed will be reliably accurate.
• Statistical analysis tests out pattern predictions based on data value distributions, and those predicted patterns under analytical testing are called hypotheses . The basic question most directly asked is generally “how reliably can I assume this hypothesis to be true?” Data quantity requirements are in this context, always about having enough of a pattern available to analyze to be able to test its ability to predict what further data would be like as to its patterns.

The next area of discussion that I will delve into is in the range and values of data obtained and I will be discussing issues such as outliers, and data distribution patterns such as normal distributions – and how different statistical tests can have data distribution requirements as well as minimal data quantity requirements. Meanwhile, you can find this and related postings at Business Strategy and Operations – 2 (and also see Business Strategy and Operations.) You can also find this at Macroeconomics and Business. As with this posting, my goal in this series is to outline and hopefully explain some of the core concepts, mathematical included, that are involved in making meaningful use of business intelligence, and in nonmathematical terms but for managers who have to be able to work with statisticians as well as with their own non-technical marketing staffs.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: