Platt Perspective on Business and Technology

Big data and the assembly of global insight out of small scale, local and micro-local data 12: data granularity and quality as costs/benefits and due diligence considerations

Posted in business and convergent technologies, reexamining the fundamentals by Timothy Platt on April 28, 2015

This is my twelfth installment to a series on big data and how wide-ranging and even globally significant insight can be developed out of small-scale local and even micro-local data (see Ubiquitous Computing and Communications – everywhere all the time 2, postings 265 and loosely following for Parts 1-11.)

I have been discussing both descriptive and predictive data analyses in Part 9, Part 10 and Part 11 of this series. And at the end of Part 11, there focusing primarily on predictive analysis and modeling, I said that I would continue its discussion here, focusing on:

• The costs and benefits dynamics of big data collection and use, and particularly where the overriding goal is to be able to address hypotheses from it with as fine a granularity as possible.

I added there, in anticipation of this posting that addressing this complex of issues of necessity raises both operational and strategic costs and risk management issues and challenges. But I begin this posting’s discussion with a more fundamental issue: that of what constitutes costs and benefits per se and with a particular focus on the costs side of that balance.

In a more strictly business context, costs and benefits are essentially always at least traditionally measured in directly fiscal and monetary terms, and entirely so. But as soon as you think through and seek to operationally and strategically address real world customers and marketplaces, you need to include in this at least some less precisely defined and measured social metrics as well, that directly impact upon and shape the values determined from those monetary evaluations. Businesses do business with people – not just with their wallets, their bank accounts and their other sources of spending power.

This holds particularly true in our current and still emerging always online and connected context where social media and the seemingly ubiquitous sharing of online reviews means any and every customer experience with essentially any providing business is going to be shared, and where that sharing can go viral – and particularly for negative reviews. This impacts upon and can enhance or limit the reputation, the competitive standing, and the sales effectiveness and competitive strength of businesses.

On one hand this means businesses needing to gather in and analyze a wider range of marketing and sales and consumer feedback into their big data accumulations, and they need to acquire more widely when they externally source this type of data. It also means conducting more complex and far-reaching data analyses, which as a matter of basic statistical reasoning requires larger and larger data samples – data from more and more customers and from more and more matched non-customer participants in the markets that a business does its business with, that would be analyzed as control groups. Much of this data can be general demographic in nature, anonymous as to individual consumer identity. But as soon as a business seeks to individually target specific customers and to pursue a demographic of one approach in marketing and selling to them, they need to broadly gather individualized data too, and in real depth. (See my series Big Data at Ubiquitous Computing and Communications – everywhere all the time, postings 177 and following for its Parts 1-7, and particularly its Part 1: the emergence of the demographic of one.)

The reason why all of this data is collected and analyzed, and often at significant direct monetary cost is to inform and to help shape strategic planning and the priorities that business strategy seeks to realize. And on an implementation level this translates directly into informing operational planning and execution too.

But a fuller range of value needs to be included and accounted for here, that goes beyond just the directly monetary plus measures that can at least relatively directly be correlated with those monetary measures. And this brings me back to a governmental case study example that I briefly touched upon in Part 11: national security-driven big data collection of essentially every type that can be gathered in about essentially everyone. (Also see my series: Learnable Lessons from Manning, Snowden and Inevitable Others atUbiquitous Computing and Communications – everywhere all the time 2, postings 227 and loosely following for its Parts 1-29.)

The more elusive but I add essential metric that I would add here to this discussion, with that working example in mind is trust. And while trust might not be easy to in any way quantify, it is vitally important and its loss can lead to very genuine long-term consequences. Trust lost or even just significantly diminished can be a long time in its recovery again. That holds in a business and commerce context, and as exemplified in this governmental policy and practice example it applies in the public sphere too.

And with this I find myself directly facing the operational and the underlying strategic reasoning behind the US government’s War on Terror-motivated comprehensive surveillance programs (see the above cited series: Learnable Lessons from Manning, Snowden and Inevitable Others and particularly its Parts 26-29 where I discuss the emerging Obama Cybersecurity doctrine and its quest for absolute security.)

• Big data collection per se, and big data use carry costs and potential costs that go way beyond the directly monetary, but that directly impact upon the organizations that develop and utilize these resources.
• Businesses that come to be seen as violating public and individual privacy face direct and financially significant consequences for that, and even if they never have to face legal action with the specific additional costs that creates as a result.
• In my public sphere example of open ended governmental surveillance in the name of national security, this carries costs too, and both through erosion of public trust in governmental leadership, and from pushback where information holding private businesses more actively seek to block outside access to their data stores, governmental access included.

With this, I come to a crucially important aspect of big data that is all too often never really considered, at least in any discussions of the underlying technologies involved there or of how they can be strategically and operationally deployed and used.

In my War on Terror driven, cyber and telephone system surveillance example, all focus seems to have been on their quest for absolute security, with all thought as to possible costs from these “non-monetary considerations” relegated to the illusory and long-term impossible task of keeping all of this completely secret forever, and no matter how wide-ranging and comprehensive this data collection effort might be.

• I hold this up as a poster child example of big data collection without any realistic awareness of or accounting for potential costs, or ultimately of realistic potentially realizable benefits either.
• Big data collection and use, and big data sharing and distribution policies can only be effective and offer more value than cost, and certainly long-term when a fuller understanding of the overall range of potential costs and benefits from them are built into the foundations of their strategic planning and policy, and into their implementations that are put in place too.

I end this series at least for now with this point, noting that I am sure come back to these and related issues in future postings and series too. Meanwhile, you can find this series and other related material at Ubiquitous Computing and Communications – everywhere all the time 2 and also in my first Ubiquitous Computing and Communications directory page. And I also include this series in my directory: Reexamining the Fundamentals.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: