Saturday, February 11, 2012

On Keeping an Eye Open At All Times

How To Keep an Eye Open Regardless of Media Channel(s):

Part I

Example of Markov Decision Process (MDP) trans...
Image via Wikipedia
In my previous post, I spoke about Content Discovery and The Enterprise, specifically about the case of dealing with flows of information external to the company, of monitoring such flows for specific signals or "intelligence". I spoke of the Gathering phase. I wanted to make a short addendum of sorts. I thought it would be interesting to note that when we talk about content "external" to the company, we are not constrained to mean only online "web content". The sources of information can be of many kinds. Any medium is as good a starting-point as another. You could do the same thing you do with web content but with newspapers, magazines, the evening news if you wanted. That's why I'm writing this short addendum, to make a few important things clear.

I always liked the way Marshall McLuhan made it sound as though everything was a "medium". All technologies were treated as media, as extensions of our body, mind, or spirit. At any rate, what the idea that most if not all things around us, our "useful objects" we use in "useful space", etc., are media, then it becomes important to "Keep an Eye Open at All Times" (i.e. "Regardless of Media Channel(s)"). But how do I do this? how does my company "Keep an Eye Open at All Times"?

There is a pretty big problem inherent to information seeking behavior in humans, I believe. You kind of have to know what you're looking for before you go looking for it. In other words, there are known unknowns and unknown unknowns. You want to stake out the space of the known unknowns at first, keeping in mind however the larger "horizon of findability", the "Big Picture".

In media and in events surrounding us, we are looking for certain signals. We try to find emerging trends, early indicators, but waning trends too. (Note: One cannot predict the future. While one is looking for "insights" or "intelligence" to aid in making optimal decisions within an organization, Content Discovery is never soothsaying.) The point is that in tracking and monitoring, we are "looking". Effectively, that is the meaning of "theory", the Greek "theoria", corresponding to the Latin "contemplatio", meaning "looking at", "gazing at", "being aware of". That is the kind of "looking" that we are doing when we speak of gathering intelligence. (And there are problems inherent in this, as we shall see. Problems that are not insurmoutable.)


NGC 346 hydrogen emission (captured by the Hub...
Image via Wikipedia
In the last post we spoke about social media intelligence. What I have to share today is that there are countless cases, types oF "intelligences". Social media intelligence is an example of intelligence dealing with information external to the company, in an information ecology we call "social media", a kind of shared "Third Place". The point is that we are not limited to one media channel; we can explore them all, internal and external. Anything that exists can be the object of study or "theoria", object of "intelligence". As I have said, however, there are problems inherent in intelligence(s). One problem is a focus problem, a problem Eli Pariser dubbed "The Filter Bubble". Another name for it might be the "Relevance Paradox". It is a case of the difficulties humans have dealing with uncertainties and unknowns, of dealing with information flows in general.

You see, the Gathering or Collection phase is all-important in any Discovery process. What you gather or collect is going to influence the results that you come up with when you analyze, synthesize, or process the gathered materials. Anything which skews or potential skews the decision process in selecting and choosing Content, in the work of Discovery, will influence the end-result. A relevance paradox occurs when one thinks he has obtained all information relevant to a decision-making process but is unaware of the existence of additional relevant information, its relevance not being apparent until he first has that information. That is why it is extremely important for an Organization, Company, or Corporation to also do an "internal content audit", to know what Data/Information/Knowledge one has to begin with. Only when one knows the exact state of one's Knowledge, we'll call it, can one know what new information one might need, i.e. through auxiliary information channels, gathering background knowledge.

When we speak of Discovery, then, it is important to consider what is our "information need", similar to what our "search intent" would be in information seeking behavior using an information retrieval system. What Eli Pariser calls "The Filter Bubble" is an important concept to remember in "Keeping an Eye Open": i.e. one must be wary of one's own biases. (See: Systemic Bias).

Any pre-existing bias in the selection process, whether it's the selection of the very object of study, or the selection of media channels to track and monitor, or what to look at in general to find insights, etc., it will skew the end-result. This means that in the end, a bias will skew the decision-making process in the corporation, the goal of Content Discovery in this case being to find insights to aid in making optimal decisions, i.e. information as decision aid.

Let there be an information space. Let there be an individual interested in gaining access to a knowledge resource. He or she follow paths, tracking "information scents", orienting and re-orienting oneself constantly in the information ecology. When one observes the very act of information seeking, and one knows the limitations of the apparatus itself of human intelligence, human judgment, sense perception, etc., one finishes by recognizing that there will inherently always be an information filtering process.

If we look at individuals, their senses and rational minds, there are fundamental limitations, one of which is it is impossible to "record" everything. There will be data loss, period, point final. It makes sense, though. We need filters. The first step in the data collection phase will essentially be to go on a data collecting "binge", and you will require the implementation of primary filters. It will help guide you through the Discovery process. That information which IS auxiliary will already be filtered by who owns the media/media channel, as well as who pays for it (advertising), etc.. (See: Herman and Chomsky's five filters, in The Propaganda Model.)

Your primary filter could be for auxiliary information, available content and its coverage, and your primary goal, to discern the shape and size of the newshole, as though it were an archaeological context you were treating/studying. Your object of study then is this context. You observe, monitor that context, noticing changes, tracking movements in the context(s), materials. You are essentially looking at physical signals and physical changes, changes in communications in this case, as content.

Step One (of data collecting binge): Scout out the terrain/territory. Define / decide on search intent (information need), i.e. Define Focus. Establish search and filter parameters. Examine ambient field, navigate ambient information space, digital space, media ecology, newshole, etc. Locate horizon of findability, of knowability, what is knowable, findable. Actively engage in exploratory search behavior, establishing peripheral awareness as you browse through the contents of the networked information ecology. End-goal: Situational Awareness/Intelligence.

Part II

Content Discovery begins with a Collection phase. We have explored methods and tools. The rest merely involves an optimization process: You iterate and optimize the overall process, gaining Direction, orienting and re-orienting, attempt to achieve specific goals, exploring the limits of the auxiliary channel(s), i.e. communication channels in this case. To counteract Information Overload problems, to reduce information noise or "contain" information pollution, you refine the filters, study your own processes in real-time, attempting after each iteration to "loosen" your own structural limitations, i.e. systemic biases. We want to shed as many biases as possible.

And so you have a battery of tools, search tools, content discovery tools, but you must actively and retroactively refine and reformulate search terms, etc. You must sharpen the Focus, i.e. sharpen the Filters too. In essence, the entire project is a refining process, an intelligence refinery whereby the tools themselves, be they psychological/rational, technical, or methodological, need to be constantly kept sharp.

As stated, problems exist which are inherent to the Discovery Enterprise itself:

  • The problem/limitations of what is knowable and findable (problem of visibility/invisibility in phenomenological ontology, the problem of credible messages vs. incredible messages in signalling games, etc., i.e. "trusted" sources, "authority".)
  • The problems inherent in the decision making process in Organizations, human dynamics of priority queuing processes, the problem of optimal decision making under extreme uncertainty, with information handicap or asymmetry, incomplete knowledge, etc.
  • The problem of native constraints of/on the imagination (the human quest for truth, meaning, beauty, purpose, sense, etc. i.e. biases in human judgment, aesthetic judgments, judgments of taste, value judgments, as exemplified by the search for what I care called "beautiful signals", signals which signal "truth and beauty".)
  • There seems to be a problem intrinsic to the enterprise of knowledge management, mainly in structuring or organizing knowledge. It can be called an urn problem, for it is a problem in stochastic urn processes which characterize numerous forms of knowledge or intelligence "refinement". In essence, you are dealing with "urns and balls", with containers and items you put in said containers. Preferential attachment processes have been known to occur here, which can explain why we encounter such things as Filter Bubbles and systemic biases such as the Relevance Paradox or Unknown Unknowns. (Semantic or ontological errors can occur as well, such as the Category Mistake / Category Error.)
The Gathering phase, though, is only the first step. Next we will look at the Processing phase, i.e. what you do with the data you have collected. We have tried to eliminate biases, errors or fallacies of logic, in documenting the space, as we transcribed the "footprint of the knowable". We have harvested data, recording facets of the news ecology, delimiting the "frontier" of its full "extension".

In the Processing phase, we will see how things get awfully "ontological", where we begin to think about classes or categories of existing things, of which "content" is an example. We will "refine" our "collection" to isolate "key quotes", quotes that are indicative of the general "whole", the content that we have gathered essentially constituting the "sample" we will work with in the Analysis/Synthesis phase.

The point of this article, though, was merely to point out that one should be extremely attentive when engaging in Content Discovery. Content exists in many forms, in many channels. Media of all shapes and sizes abound. We are essentially always immersed in an immense field of information. It is therefore to our advantage to "Keep an Eye Open at All Times, Regardless of Media Channel(s)".

This means that we must be ever careful. The auxiliary information corporations are currently looking for might not be the most pertinent or relevant in helping them make the best decisions. Knowledge is funny that way. You don't really know what you are looking for, at the outset. You know one thing, you are looking for Signals. To find them, these "golden nuggets of truth", these "beautiful signals", signals which excite your senses, which enlighten and edify, which strengthen convictions and help elucidate problems and elaborate solutions, to find them you need to "separate the wheat from the chaff", as the idiom goes. This entails an information filtering process.

It is to our advantage therefore to develop "robust" methods, "elastic" methods that we can alter and refine over time, adapting to the changing networked information /media ecology in real-time. We want to fail quickly, and iterate, learning from our mistakes, our "design failures". Our end-goal is Resilience, Fitness, Soundness, of reasonings and methodologies. This entails finding ways to innoculate ourselves against our own systemic biases, thereby avoiding Filter Bubbles and Filter Failure, trying to limit the influence of our own aesthetic judgments or judgments of taste, our value judgments too, which tend to adversely create constraints on the will and the imagination, limiting the extent of the "possible", because beauty, truth, and meaning are powerful filters.. Beauty mesmerizes, locks us in a state of awe and stupefaction. Beauty and Truth can cloud the senses. They are to be watched closely.

Keeping an Eye Open at All Times means observing one's own information seeking behaviors, to better understand the field of one's own behavioral ecology, such as identifying social information or collaborative information seeking behaviors, ways that we naturally interact with information flows. The purity of your methods will help you the most in the long-run. If you actively seek to limit the extent of your own biases, you will find that the "sample" you collect is more representative. Your stance towards the information space must be indirect, impersonal, one of distance and disinterestedness. Or else your own "gluttony" or "covetousness" will cloud your senses, skewing the data in favor of some figment of your imagination, some nonexistent shadow you are chasing. This is the definition of aegri somnia, a sick man's dreams. You are seeking signals, market signals, physical signals, credible signs. But you must fight against your own faculties of judgment, for human judgment, while it is an expert in categorizing and organizing knowledge, has a nasty habit of creating beautiful proofs and beautiful answers, in the face of uncertainty, when there is a void of sense, man can actually create them. You must erase yourself as much as possible from the process, so that your own presence does not influence the footprint of the known and knowable. This lesson is of utmost importance in the work of intelligence analysis, of which we will speak about at length in future articles.

Remember, you are wayfinding, not trying to erect a Tower of Babel. A measure of Skepticism, Doubt, and Restraint is in order, as well as methodological rigor.

Enhanced by Zemanta

Friday, January 6, 2012

Content Discovery and The Enterprise


There are two kinds of Content Discovery for Enterprises: INTERNAL and EXTERNAL. Internal Content Discovery has to do with every piece of "Content" that the Enterprise "owns"; every piece of "media", from Invoices to Spreadsheets to Documents to Emails to Digital Images. Anything and everything that is communicated or shared between anyone within an Organization is content that can be "discovered". This can be done through some form of "content audit" and a vast body of literature exists on the subject (for starters, see: Discovery - for legal its legal counterpart - and Digital Asset Management, Enterprise Content Management, Information Lifecycle Management, and so forth). I want to talk about External Content Discovery where we are dealing with digital objects and their informational content, not their intrinsic value, or digital artifactual value.

Image representing EQENTIA - as depicted in Cr...
Image via CrunchBase
There are many reasons a Company might want to go foraging for content outside its four walls. Usually, the purpose is to acquire some kind of competitive or business intelligence. It can take place anywhere that Content, i.e. any "media" whatsoever, any piece of communication whatsoever, can be found, but for our purposes we are monitoring the World Wide Web, one of the richest sources of content pertinent to corporations and their strategic foresight. There are many ways to "monitor" the web; a virtual panoply of tools exist, from newsreader applications to more formal "discovery engines", tools for acquiring "social media intelligence".

It's up to these newly minted "discovery agents" to build a Content Discovery grand strategy and as was indicated, content discovery tools abound. A few examples are Trap.it, Eqentia, Trove, Zite, Google Reader, etc. Twitter, Facebook, and Google+ are also great sources of Content relevant to your Organization. But if all these "tools" frighten you, you really need nothing more than a organic, vanilla search engine, and for that Google Search, though not the best, will certainly suffice.
Image representing Trapit as depicted in Crunc...
Image via CrunchBase

As previously mentioned, there are many reasons or "purposes" for engaging in Content Discovery outside the Enterprise. Usually, though, its purpose is to find "auxiliary information", "external knowledge", "side information", that can be collected and analyzed and used to provide some form of "intelligence" that will aid the Company in question in their quest to Innovate and stay Competitive or "Cutting Edge". It aims to attain a situational understanding or assessment through "Situational Awareness".

Content Discovery involves the Gathering of information, i.e. "Content". Content is any type of medium whatsoever, as stated before. The Collection phase of the Discovery process involves a) finding credible sources of information, b) collecting content or information from those sources and there is a third phase c) what you do once you've amassed said collection (which involves various forms of storing and organizing, classifying, data analysis, social data analysis, content analysis, business intelligence, collaborative intelligence, predictive analytics, etc.

In essence, the Gathering phase involves any form of what can be called "Environmental Scanning":

I will leave it at the Gathering phase for now and continue in a second article which will focus on the Relevance Paradox in decision-making / decision processes in certain corporations. If you enjoyed this article and what to pursue your readings, there may be something of worth to you in an old article I wrote on market signals which can be found here. The Market for Scepticism + The Virtue of Failure

In the next article or series of articles, I will talk more about the kinds of auxiliary information corporations are currently looking for, and how they may not be making optimal decisions in what I call the priority queue, or queuing process. So we look at suboptimal cases of that. I will get into market signals, social media intelligence, the brand graph, scepticism in business, failure analysis, how failure contributes to the advancement of Design Science, a bit on Taxonomies and Data Analysis.

Hope that you are well. Take care, thanks for stopping bye, and keep stay tuned - more is coming. If you want to get in touch, many channels are available.

You can find me here:
On Twitter: @jonasthanatos

Enhanced by Zemanta

Wednesday, November 16, 2011

Rediscovering Discovery: The Daily Digital Curation of Beautiful Signals

The Daily Digital Curation of Beautiful Signals 
Kantian Aesthetics and The Information Highway

The official verdict is out. Surfing the web for relevant content or in the hopes of serendipitous content discovery IS an Aesthetic Experience, period. That is, we are looking for information-objects of beauty. The experience of information serendipity is akin to the sublime in art. I explain.
We are hearing more and more being spoken with regard to relevance, content discovery, and the assuagement of one's information needs. We also hear much on the subject of information noise. What is information noise?

Well, we know what noise is. Currently, our definition of information noise is usually UNWANTED INFORMATION, i.e. unwanted information present in your social stream. So in a sense, YOU are NOISE to me.

Septem artes liberales from "Hortus delic...
Image via Wikipedia
That is to say, there are Twitter users whose posts I don't like reading: They are not beautiful, hence they are information pollution / information noise. We even speak of "cleaning" one's Twitter Feed.So we're speaking of an aesthetic experience, and the way I see it, when we speak of cleaning our social stream, or keeping it tidy, we're really speaking about social standing.

What do I mean by this? What I mean is that we're trying to control what kind of information reaches us, partly by cleaning up what digital channels will NOT reach us. We are choosing kinds of information that we do NOT want to see in our social stream.

In a sense, we are also categorizing PEOPLE in terms of Sound information and Noisy information. We are therefore classifying people as being either Good or Not Good (i.e. Good or not Good FOR ME).
So a different kind of filtering is taking place. We're not just talking about information filtering anymore: We're talking about an aesthetic filtering of PEOPLE, because PEOPLE in the end are attached to these Twitter or Facebook or other accounts.

It's normal to want to control one's "listening station", to get the kinds of information we prefer, and none can doubt that surfing the web for information is a matter of taste. But it becomes an aesthetic problem and a very serious ethical debate arises out of it.

How can I judge a human being as being either informative or as being information noise? I can classify INDIVIDUALS as satisfying my information needs or NOT satisfying my information needs. This is horrible, though, the consequences are terrifying. I can dispose of individuals as though they were garbage, and believe me, the effect is felt. You try being rejected as a POOR SOURCE of information. You will be ignored, rejected, put on the Black List, and you will feel alienated.

So for the most part, this kind of "social filtering" of people is really unethical. We speak of our egalitarian society as righteous and virtuous. Yet we classify and label people as good and not good. Noise, in the digital information channel, is bad. It is UGLY. It is a matter of taste, but when taste labels & judges individuals, it becomes a serious ethical problem of VIOLATION and TRANSGRESSION.

Beham, (Hans) Sebald (1500-1550): Dialectica (...
Image via Wikipedia
Just as a piece of information that satisfies my information need is seen as RELEVANT and BEAUTIFUL, it also transgresses the norm of what is UGLY and NOISY and NOT USEFUL, or NOT INTERESTING. The same goes for the uninteresting, it violates that value we hold so dear, those values indelibly linked to the labelling of our individual experiences of the Information Highway as beautiful, true, and good.

Yes, we are actually seeking the Good, the Beautiful and the True, and are labelling people this way in the process. But an individual is neither good, nor beautiful, nor true, nor is he or she the opposite of this. Humans are just what they are. They can be annoying, but really when one takes offense, one is usually looking to be offensed.

So this, for me, is the final verdict on the experience of surfing the World Wide Web. It is a matter of aesthetics and aesthetic taste, and a seriously unethical matter of FILTERING PEOPLE as objects of knowledge. And so a question arises with regard to the liberal art of Saving-Face: To put it politely, we're looking for social standing, and the judgment of others, as to our aesthetic beauty, goodness, or truth, is profoundly important to us. Services such as Facebook permit us to gauge the responses others have of us, so it works to keep us in check. Like the Liberal Arts in the 17th and 18th century in England, the Liberal Arts serve a public function of promoting the civic virtues. We are doing the same at present: We are using each other to promote civic virtues. Relevance and Utility have become civic virtues, and others can take on the Face-Wasting role of lacking in civility. More to come on the subject of Face-Saving and Face-Wasting (i.e. The Antiface Strategy).


Golden Radio: The Beautiful Signal God

I once liked to think of irrelevant, uninteresting or unwanted information as "noise" in language media / communications media.. But then people started using the term signal-to-noise ratio and I thought the analogy was taken too far. Really if it is information that is impertinent, just call it impertinent. If it has to do with the level of background noise, say level of background noise. I don't like it when metaphors or analogies are used merely to save time. Like instead of explaining the intricacies of the Trinitarian concept of God in Catholic doctrine, you use an analogy like a three-leaved clover.. it can be pretty even cute, but I find such time-saving shortcut metaphors to be awfully "noisy" themselves.. So I try to ignore it.. and when I'm not listening,I fall into silent contemplation again..

See Wikipedia: en.wikipedia.org/wiki/Signal_to_noise_ratio

"Signal-to-noise ratio (often abbreviated SNR or S/N) is a measure used in science and engineering to quantify how much a signal has been corrupted by noise. It is defined as the ratio of signal power to the noise power corrupting the signal. A ratio higher than 1:1 indicates more signal than noise. While SNR is commonly quoted for electrical signals, it can be applied to any form of signal (such as isotope levels in an ice core or biochemical signaling between cells).
In less technical terms, signal-to-noise ratio compares the level of a desired signal (such as music) to the level of background noise. The higher the ratio, the less obtrusive the background noise is.
 
"Signal-to-noise ratio" is sometimes used informally to refer to the ratio of useful information to false or irrelevant data in a conversation or exchange. For example, in online discussion forums and other online communities, off-topic posts and spam are regarded as "noise" that interferes with the "signal" of appropriate discussion."


A New Signal

I've been thinking about signals a lot and for a very long time. I've been obsessed with signals since I started playing the electric guitar about twenty years ago. I eventually studied sound design and my love of signals has only grow with time.

So naturally I met the new discourse around signals & noise in information technologies with eagerness and an open heart. Many of us have been following Social Streams now for quite some time, and the discussion over the signal-to-noise ratio is currently raging strong.

The Seven liberal arts. Music and Pythagoras.
Image via Wikipedia
I began thinking about an hour or so ago about different signal types. When I record audio signals, whether it's my voice or my electric guitar, I work with analog signals usually streaming through electronics equipment which I then digitize, ending up with digital signals.

What we are calling Signals in computer-mediated communication is closer to the definition of "market signals", which are a form of Information. At any rate, I've been thinking of these information streams and trying to think of what the next type of signal might be.

Let's face it, we're mostly dealing with a stream of information, mostly in the form of links to media or links to profiles, blogs, various other kinds of dynamic websites, etc. A lot of the time, too, on sites such as Facebook and Twitter, we're still seeing a stream of Status Updates.

Between a stream of equal part links and status updates, I wonder about what new kinds of streaming information might occur in the near future. We have people following the stock market and the kinds of information found therein. A lot of what is streaming we could just call News. Some of it is personal information, information about various "states of affairs".

So I have to ask you, what new kind of signals do you think might appear in the near future? Maybe machine-to-machine signals in the form of what is called electronic negotiation? In other words, your washing machine might negotiate with energy sources to optimize the electrical signal being used by the machine, or the other appliances in the house, negotiating with the price of sources per kilowatt-milliseconds.

Human Information Behavior, Information Noise, and Social Stream Filtering

I've been thinking a great deal about information filtering, particularly social filtering and noise filtering too. In sound design, I'm used to hearing terms like "noise filtering" and the famous "signal-to-noise ratio". In recent times, however, I've seen the term noise used with regard to information overload / information pollution and the term signal-to-noise ratio used with regard to social streams.

In digital signal processing, we can speak of "low-pass filters" or "high-pass filters", which made me think of the "pertinence-pass filter" that could exist for filtering social streams.

I guess you'd have to equate relevance with a given frequency or frequency range.

I've also been thinking of black noise, i.e. Black Noise and Population Persistence, as well as Noise Trading.

So many thoughts, so much to synthesize. I'm thinking of ESS, Survivalism, Evolutionary Learning Algorithms, Negative Filtering (filtering "out", i.e. opting-out what you don't want, Digg's "Bury" button).
I want a system that can learn my information preferences. It would be requisite that information gathered, i.e. the dataset, have appropriate metadata, so that filtering agents could filter through metadata-rich data, through some sort of sorting algorithm, "listening" for my preference / attention profile.

I'm also thinking of bioelectronics and Noise Servers, Noise Modelling, Matching, Signal Purification, Filter Synthesis, and Human Information Behavior (also: Collaborative Information Behavior, i.e. Collaborative Information Seeking. The point is that sound designers need to get together with computer programmers and solve the social stream filtering problem once and for all. Just treat it as noise and use digital signal processing algorithms to "cancel out" or "remove" what is a noise in the "signals". We should also begin treating social streams as signals, because that's what they are.
Enhanced by Zemanta