Platt Perspective on Business and Technology

Meaning in information theory – a reconsideration 1

Posted in macroeconomics by Timothy Platt on July 13, 2012

When Claude Shannon and Andrey Kolmogorov independently began working on and developing mathematical theories of information they made a same crucially important axiomatic assumption underlying their work. They both looked at sequences of data and at the patterns inherent in them, divorced from any collateral conception of meaning (see information theory.)

• Mathematically at least, information became a matter of pattern and predictability.
• Sequences were determined as being random or as being calculable and non-random.
• And sequences of interest were identifiable as being capable of reversible compression through their capacity for being calculated by an algorithm shorter than their original data sequence.

Properties of this type were becoming increasingly important in an engineering and information processing and transmission context. The microphones and broadcast equipment, transmission channels and radio receivers of Shannon’s and Kolmogorov’s time, that they were thinking in terms of when first developing information theory, do not have to understand the information they process and convey. And neither do telegraph system technologies already long in place or the newer electronic processing, storage and transmission systems that have been developed since then. They all just have to be able to process information and carry it and at seemingly ever-increasing rates and volumes, and deliver it all at a distance with its content intact, or at least with a minimal amount of signal distortion and other error sources added in so as to preserve meaning and fidelity. But even as information theory focuses on information processing and flow, and on information volume we still all tend to think of information primarily in terms of content and meaning.

I write about information volume and about big data in the course of posting to this blog (e.g. see Massive Databases, Cloud Computing and the Killer Data App.) But my focus is more on business intelligence and on what that information means – and certainly when I write of its monetizable value (see for example, Macroeconomics and Business, postings 3, 9 and 21-32.) And I find myself thinking back to the limitations of meaning that are carried in mathematical information theory.

Information’s meaning is removed from consideration because it is so difficult to include it and still be able to mathematically process and analyze information. Meaning as a concept does not readily fit into the restrictions of simple, calculable, unambiguous algorithmic patterns. A single data sequence can hold one meaning to one person and another to a different person, and perhaps no real meaning at all to a third.

• Is 1492 the year that Columbus discovered the New World?
• Is it the year that Columbus first made contact with the people indigenous to the lands he was “discovering”, as he labored under the delusion that he was in fact on the shores of the Indian Subcontinent and in a very different place?
• Is 1492 a brand name of rum, or of an aftershave?
• Is it simply a number that happens to be divisible by 2 and 373?

Needless to say, that brief list of possibilities only begins to touch the surface, and particularly where idiosyncratic meanings are added in, such as the street number of a first home owned, or a utilized ATM machine pin number. Meaning was seen as too messy and ambiguous for inclusion in anything like a rigorous mathematical theory of information.

But as a thought piece I have decided to at least begin a discussion as to how one part of the meaning puzzle might be mathematically included there. And I begin by making a simple, basic observation:

• Information’s meaning is derived from its context.

And by that I mean all meaning is context based. Looking back at the 1492 example, different people see different meaning in that same perhaps seemingly simple number because they see it from the various contexts of their own educations and their own personal histories. A “none of the above” for that number might be “1492 is the year the Spanish Crown expelled all Jews and Muslims from their country, and by edict from the entire Hispanic Peninsula.” This is all about context, and the contexts we carry with us. So any attempt to mathematically define or delimit meaning, would have to involve more rigorously defining context. And as a first, small step in that direction I would propose a loosely defined test for determining if there is a significant context that could be considered a foundation point for identifying meaning in a data sequence.

• If combining a new sequence of information data with and into an a priori context set of data permits a collective description by algorithm that is smaller than the sum of the two sequences: new plus context, then the new sequence under consideration can be said to hold meaning in the context of that a priori data sequence it is juxtaposed and combined with.

I add as an immediate corollary that:

• The greater the compression – the greater the savings in information string length from combining new sequence and context data, the greater the value of the meaning that this new sequence carries and brings to that context.

This definition would apply to information data strings that are themselves subject to unambiguous mathematical description, and for which algorithmic descriptions are in some way possible. So this is only a small first step. But I expect to come back to this in future postings. Meanwhile, you can find this and related postings at Macroeconomics and Business.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: