Integrated information theory

File:Phi-iit-symbol.svg

Phi, the symbol for integrated information.

Integrated information theory (IIT) attempts to explain what consciousness is and why it might be associated with certain physical systems. Given any such system, the theory predicts whether that system is conscious, to what degree it is conscious, and what particular experience it is having (see Central Identity). According to IIT, a system's consciousness is determined by its causal properties and is therefore an intrinsic, fundamental property of certain causal systems.

IIT was proposed by neuroscientist Giulio Tononi in 2004, and has been continuously developed over the past decade. The latest version of the theory, labeled IIT 3.0, was published in 2014.

Overview

Relationship to the "Hard Problem of Consciousness"

David Chalmers has argued that any attempt to explain consciousness in purely physical terms (i.e. to start with the laws of physics as they are currently formulated and derive the necessary and inevitable existence of consciousness) eventually runs into the so-called "hard problem". Rather than try to start from physical principles and arrive at consciousness, IIT "starts with consciousness" (accepts the existence of consciousness as certain) and reasons about the properties that a postulated physical substrate would have to have in order to account for it. The ability to perform this jump from phenomenology to mechanism rests on IIT's assumption that if a conscious experience can be fully accounted for by an underlying physical system, then the properties of the physical system must be constrained by the properties of the experience.

Specifically, IIT moves from phenomenology to mechanism by attempting to identify the essential properties of conscious experience (dubbed "axioms") and, from there, the essential properties of conscious physical systems (dubbed "postulates").

Axioms: essential properties of experience

The axioms are intended to capture the essential aspects of every conscious experience. Every axiom should apply to every possible experience.

The wording of the axioms has changed slightly as the theory as developed, and the most recent and complete statement of the axioms is as follows:

* Intrinsic existence: Consciousness exists: each experience is actual—indeed, that my experience here and now exists (it is real) is the only fact I can be sure of immediately and absolutely. Moreover, my experience exists from its own intrinsic perspective, independent of external observers (it is intrinsically real or actual).

• Composition: Consciousness is structured: each experience is composed of multiple phenomenological distinctions, elementary or higher-order. For example, within one experience I may distinguish a book, a blue color, a blue book, the left side, a blue book on the left, and so on.
• Information: Consciousness is specific: each experience is the particular way it is—being composed of a specific set of specific phenomenal distinctions—thereby differing from other possible experiences (differentiation). For example, an experience may include phenomenal distinctions specifying a large number of spatial locations, several positive concepts, such as a bedroom (as opposed to no bedroom), a bed (as opposed to no bed), a book (as opposed to no book), a blue color (as opposed to no blue), higher-order “bindings” of first-order distinctions, such as a blue book (as opposed to no blue book), as well as many negative concepts, such as no bird (as opposed to a bird), no bicycle (as opposed to a bicycle), no bush (as opposed to a bush), and so on. Similarly, an experience of pure darkness and silence is the particular way it is—it has the specific quality it has (no bedroom, no bed, no book, no blue, nor any other object, color, sound, thought, and so on). And being that way, it necessarily differs from a large number of alternative experiences I could have had but I am not actually having.
• Integration: Consciousness is unified: each experience is irreducible to non-interdependent, disjoint subsets of phenomenal distinctions. Thus, I experience a whole visual scene, not the left side of the visual field independent of the right side (and vice versa). For example, the experience of seeing the word “BECAUSE” written in the middle of a blank page is irreducible to an experience of seeing “BE” on the left plus an experience of seeing “CAUSE” on the right. Similarly, seeing a blue book is irreducible to seeing a book without the color blue, plus the color blue without the book.
• Exclusion: Consciousness is definite, in content and spatio-temporal grain: each experience has the set of phenomenal distinctions it has, neither less (a subset) nor more (a superset), and it flows at the speed it flows, neither faster nor slower. For example, the experience I am having is of seeing a body on a bed in a bedroom, a bookcase with books, one of which is a blue book, but I am not having an experience with less content—say, one lacking the phenomenal distinction blue/not blue, or colored/not colored; or with more content—say, one endowed with the additional phenomenal distinction high/low blood pressure. Moreover, my experience flows at a particular speed—each experience encompassing say a hundred milliseconds or so—but I am not having an experience that encompasses just a few milliseconds or instead minutes or hours.
  — Dr. Giulio Tononi, Integrated information theory, Scholarpedia

Postulates: properties required of the physical substrate

The axioms describe regularities in conscious experience, and IIT seeks to explain these regularities. What could account for the fact that every experience exists, is structured, is differentiated, is unified, and is definite? IIT argues that the existence of an underlying causal system with these same properties offers the most parsimonious explanation. Thus a physical system, if conscious, is so by virtue of its causal properties.

The properties required of a conscious physical substrate are called the "postulates," since the existence of the physical substrate is itself only postulated (remember, IIT maintains that the only thing one can be sure of is the existence of one's own consciousness). In what follows, a "physical system" is taken to be a set of elements, each with two or more internal states, inputs that influence that state, and outputs that are influenced by that state (neurons or logic gates are the natural examples). Given this definition of "physical system", the postulates are:

* Intrinsic existence: To account for the intrinsic existence of experience, a system constituted of elements in a state must exist intrinsically (be actual): specifically, in order to exist, it must have cause-effect power, as there is no point in assuming that something exists if nothing can make a difference to it, or if it cannot make a difference to anything. Moreover, to exist from its own intrinsic perspective, independent of external observers, a system of elements in a state must have cause-effect power upon itself, independent of extrinsic factors. Cause-effect power can be established by considering a cause-effect space with an axis for every possible state of the system in the past (causes) and future (effects). Within this space, it is enough to show that an “intervention” that sets the system in some initial state (cause), keeping the state of the elements outside the system fixed (background conditions), can lead with probability different from chance to its present state; conversely, setting the system to its present state leads with probability above chance to some other state (effect).

• Composition: The system must be structured: subsets of the elements constituting the system, composed in various combinations, also have cause-effect power within the system. Thus, if a system ABC is constituted of elements A, B, and C, any subset of elements (its power set), including A, B, C, AB, AC, BC, as well as the entire system, ABC, can compose a mechanism having cause-effect power. Composition allows for elementary (first-order) elements to form distinct higher-order mechanisms, and for multiple mechanisms to form a structure.
• Information: The system must specify a cause-effect structure that is the particular way it is: a specific set of specific cause-effect repertoires—thereby differing from other possible ones (differentiation). A cause-effect repertoire characterizes in full the cause-effect power of a mechanism within a system by making explicit all its cause-effect properties. It can be determined by perturbing the system in all possible ways to assess how a mechanism in its present state makes a difference to the probability of the past and future states of the system. Together, the cause-effect repertoires specified by each composition of elements within a system specify a cause-effect structure. [...]
• Integration: The cause-effect structure specified by the system must be unified: it must be intrinsically irreducible to that specified by non-interdependent sub-systems obtained by unidirectional partitions. Partitions are taken unidirectionally to ensure that cause-effect power is intrinsically irreducible - from the system’s intrinsic perspective - which implies that every part of the system must be able to both affect and be affected by the rest of the system. Intrinsic irreducibility can be measured as integrated information (“big phi” or ${\textstyle \Phi}$, a non-negative number), which quantifies to what extent the cause-effect structure specified by a system’s elements changes if the system is partitioned (cut or reduced) along its minimum partition (the one that makes the least difference). By contrast, if a partition of the system makes no difference to its cause-effect structure, then the whole is reducible to those parts. If a whole has no cause-effect power above and beyond its parts, then there is no point in assuming that the whole exists in and of itself: thus, having irreducible cause-effect power is a further prerequisite for existence. This postulate also applies to individual mechanisms: a subset of elements can contribute a specific aspect of experience only if their combined cause-effect repertoire is irreducible by a minimum partition of the mechanism (“small phi” or ${\textstyle \varphi}$).
• Exclusion: The cause-effect structure specified by the system must be definite: it is specified over a single set of elements—neither less nor more—the one over which it is maximally irreducible from its intrinsic perspective (${\textstyle \Phi^{Max}}$), thus laying maximal claim to intrinsic existence. [...] With respect to causation, this has the consequence that the “winning” cause-effect structure excludes alternative cause-effect structures specified over overlapping elements, otherwise there would be causal overdetermination. [...] The exclusion postulate can be said to enforce Occam’s razor (entities should not be multiplied beyond necessity): it is more parsimonious to postulate the existence of a single cause-effect structure over a system of elements—the one that is maximally irreducible from the system’s intrinsic perspective—than a multitude of overlapping cause-effect structures whose existence would make no further difference. The exclusion postulate also applies to individual mechanisms: a subset of elements in a state specifies the cause-effect repertoire that is maximally irreducible (MICE) within the system (${\textstyle \Phi^{Max}}$), called a core concept, or concept for short. Again, it cannot additionally specify a cause-effect repertoire overlapping over the same elements, because otherwise the difference a mechanism makes would be counted multiple times. [...] Finally, the exclusion postulate also applies to spatio-temporal grains, implying that a conceptual structure is specified over a definite grain size in space (either quarks, atoms, neurons, neuronal groups, brain areas, and so on) and time (either microseconds, milliseconds, seconds, minutes, and so on), the one at which ${\textstyle \Phi}$ reaches a maximum. [...] Once more, this implies that a mechanism cannot specify a cause-effect repertoire at a particular temporal grain, and additional effects at a finer or coarser grain, otherwise the differences a mechanism makes would be counted multiple times.
  — Dr. Giulio Tononi, Integrated information theory, Scholarpedia

Mathematics: formalization of the postulates

For a complete and thorough account of the mathematical formalization of IIT, see. What follows is intended as a brief summary, adapted from, of the most important quantities involved. Pseudocode for the algorithms used to calculate these quantities can be found at.

A system refers to a set of elements, each with two or more internal states, inputs that influence that state, and outputs that are influenced by that state. A mechanism refers to a subset of system elements. The mechanism-level quantities below are used to asses the integration of any given mechanism, and the system-level quantities are used to asses the integration of sets of mechanisms ("sets of sets").

In order to apply the IIT formalism to a system, its full transition probability matrix (TPM) must be known. The TPM specifies the probability with which any state of a system transitions to any other system state. Each of the following quantities is calculated in a bottom-up manner from the system's TPM.

Mechanism-level quantities
A cause-effect repertoire ${\textstyle CER(m_t, Z_{t\pm1})=\{p_{cause}(z_{t-1}|m_t), p_{effect}(z_{t+1}|m_t)\}}$ is a set of two probability distributions, describing how the mechanism ${\textstyle M_t}$ in its current state ${\textstyle m_t}$ constrains the past and future states of the sets of system elements ${\textstyle Z_{t-1}}$ and ${\textstyle Z_{t+1}}$, respectively. Note that ${\textstyle Z_{t-1}}$ may be different from ${\textstyle Z_{t+1}}$, since the elements that a mechanism affects may be different from the elements that affect it.
A partition ${\textstyle P = \{M_1, Z_1;M_2,Z_2\}}$ is a grouping of system elements, where the connections between the parts ${\textstyle \{M_1,Z_1\}}$ and ${\textstyle \{M_2,Z_2\}}$ are injected with independent noise. For a simple binary element ${\textstyle A}$ which outputs to a simple binary element ${\textstyle B}$, injecting the connection ${\textstyle A \to B}$ with independent noise means that the input value which ${\textstyle A}$ receives, ${\textstyle 0}$ or ${\textstyle 1}$, is entirely independent of the actual state of ${\textstyle B}$, thus rendering ${\textstyle B}$ causally ineffective. ${\textstyle P_{t\pm1}}$ denotes a pair of partitions, one of which is considered when looking at a mechanism's causes, and the other of which is considered when looking at its effects.
The earth mover's distance ${\textstyle EMD(p_1, p_2)}$ is used to measure distances between probability distributions ${\textstyle p_1}$ and ${\textstyle p_2}$. The EMD depends on the user's choice of ground distance between points in the metric space over which the probability distributions are measured, which in IIT is the system's state space. When computing the EMD with a system of simple binary elements, the ground distance between system states is chosen to be their Hamming distance.
Integrated information ${\textstyle \varphi}$ measures the irreducibility of a cause-effect repertoire with respect to partition ${\textstyle P_{t\pm1}}$, obtained by combining the irreducibility of its constituent cause and effect repertoires with respect to the same partitioning. The irreducibility of the cause repertoire with respect to ${\textstyle P_{t-1}}$ is given by ${\textstyle \varphi_{cause}(m_t, Z_{t-1}, P_{t-1}) = EMD(p_{cause}(z_{t-1}|m_t), p_{cause}(z_{1,t-1}|m_{1,t}) \times p_{cause}(z_{2,t-1}|m_{2,t}))}$, and similarly for the effect repertoire. Combined, ${\textstyle \varphi_{cause}}$ and ${\textstyle \varphi_{effect}}$ yield the irreducibility of the ${\textstyle CER}$ as a whole: ${\textstyle \varphi(m_t,Z_{t\pm1},P_{t\pm1}) = \min(\varphi_{cause}(m_t, Z_{t-1}, P_{t-1}), \varphi_{effect}(m_t, Z_{t+1}, P_{t+1})).}$.
The minimum-information partition of a mechanism and its purview is given by ${\textstyle MIP(m_t, Z_{t\pm1})=\operatorname*{arg\,min}_{P_{t\pm1}} \, (\varphi(m_t,Z_{t\pm1},P_{t\pm1}))}$. The minimum-information partition is the partitioning that least affects a cause-effect repertoire. For this reason, it is sometimes called the minimum-difference partition. Note that the minimum-information "partition", despite its name, is really a pair of partitions. We call these partitions ${\textstyle MIP_{cause}}$ and ${\textstyle MIP_{effect}}$.
There is at least one choice of elements over which a mechanism's cause-effect repertoire is maximally irreducible (in other words, over which its ${\textstyle \varphi}$ is highest). We call this choice of elements ${\textstyle Z^*_{t\pm1}=\{Z^*_{t-1},Z^*_{t+1}\}}$, and say that this choice specifies a maximally irreducible cause-effect repertoire. Formally, ${\textstyle Z^*_{t-1} = \{\operatorname*{arg\,max}_{Z_{t-1}} \, (\varphi_{cause}(m_t, Z_{t-1}, MIP_{cause}))\}}$ and ${\textstyle Z^*_{t+1} = \{\operatorname*{arg\,max}_{Z_{t+1}} \, (\varphi_{effect}(m_t, Z_{t+1}, MIP_{effect}))\}}$.
The concept ${\textstyle CER(m_t,Z^*_{t\pm1})=\{p_{cause}(z^*_{t-1}|m_t),p_{effect}(z^*_{t+1}|m_t)\}}$ is the maximally irreducible cause-effect repertoire of mechanism${\textstyle M_t}$ in its current state ${\textstyle m_t}$ over ${\textstyle Z^*_{t\pm1}}$, and describes the causal role of ${\textstyle M_t}$ within the system. Informally, ${\textstyle Z^*_{t\pm1}}$ is the concept's purview, and specifies what the concept "is about". The intrinsic cause-effect power of ${\textstyle m_t}$ is the concept's strength, and is given by: ${\textstyle \varphi^{Max}(m_t) = \varphi(m_t,Z^*_{t\pm1},MIP) = min(\varphi_{cause}(m_t,Z^*_{t-1},MIP_{cause}),\varphi_{effect}(m_t,Z^*_{t+1},MIP_{effect}))}$

System-level quantities
A cause-effect structure ${\textstyle C(s_t)}$ is the set of concepts specified by all mechanisms with ${\textstyle \varphi^{Max}(m_t) > 0}$ within the system ${\textstyle S_t}$ in its current state ${\textstyle s_t}$. If a system turns out to be conscious, its cause-effect structure is often referred to as a conceptual structure.
A unidirectional partition ${\textstyle P_{\to} = \{S_1,S_2\}}$ is a grouping of system elements where the connections from the set of elements ${\textstyle S_1}$ to ${\textstyle S_2}$ are injected with independent noise.
The extended earth mover's distance ${\textstyle XEMD(C_1, C_2)}$ is used to measure the minimal cost of transforming cause-effect structure ${\textstyle C_1}$ into structure ${\textstyle C_2}$. Informally, one can say that–whereas the EMD transports the probability of a system state over the distance between two system states–the XEMD transports the strength of a concept over the distance between two concepts. In the XEMD, the "earth" to be transported is intrinsic cause-effect power (${\textstyle \varphi^{Max}}$), and the ground distance between concepts ${\textstyle A}$ and ${\textstyle B}$ with cause repertoires ${\textstyle A_{cause}}$ and ${\textstyle B_{cause}}$ and effect repertoires ${\textstyle A_{effect}}$ and ${\textstyle B_{effect}}$ is given by ${\textstyle EMD(A_{cause}, B_{cause}) + EMD(A_{effect}, B_{effect})}$.
Integrated (conceptual) information ${\textstyle \Phi(s_t, P_{\to}) = XEMD(C(s_t)|C(s_t,P_{\to}))}$ measures the irreducibility of a cause-effect structure with respect to a unidirectional partition. ${\textstyle \Phi}$ captures how much the cause-effect repertoires of the system's mechanisms are altered and how much intrinsic cause effect power (${\textstyle \varphi^{Max}}$) is lost due to partition ${\textstyle P_{\to}}$.
The minimum-information partition of a set of elements in a state is given by ${\textstyle MIP(s_t)=\operatorname*{arg\,min}_{P_\to} \, (\Phi(s_t,P_{\to}))}$. The minimum-information partition is the unidirectional partition that least affects a cause-effect structure ${\textstyle C(s_t)}$.
The intrinsic cause-effect power of a set of elements in a state is given by ${\textstyle \Phi^{Max}(s^*_t) = \Phi(s^*_t, MIP(s^*_t)) }$, such that for any other ${\textstyle S_t}$ with ${\textstyle (S_t \cap S^*_t) \neq \emptyset}$, ${\textstyle \Phi(s_t) \leq \Phi(s^*_t)}$. According to IIT, a system's ${\textstyle \Phi^{Max}}$ is the degree to which it can be said to exist.
A complex is a set of elements ${\textstyle S^*_t}$ with ${\textstyle \Phi^{Max} = \Phi(s^*_t) > 0}$, and thus specifies a maximally irreducible cause-effect structure, also called a conceptual structure. According to IIT, complexes are conscious entities.

Cause-effect space

For a system of $N$ simple binary elements, cause-effect space is formed by $2*2^N$ axes, one for each possible past and future state of the system. Any cause-effect repertoire $R$, which specifies the probability of each possible past and future state of the system, can be easily plotted as a point in this high-dimensional space: The position of this point along each axis is given by the probability of that state as specified by $R$. If a point is also taken to have a scalar magnitude (which can be informally thought of as the point's "size", for example), then it can easily represent a concept: The concept's cause-effect repertoire specifies the location of the point in cause-effect space, and the concept's $\varphi^{Max}$ value specifies that point's magnitude.

In this way, a conceptual structure $C$ can be plotted as a constellation of points in cause-effect space. Each point is called a star, and each star's magnitude ($\varphi^{Max}$) is its size.

Central Identity

IIT addresses the mind-body problem by proposing an identity between phenomenological properties of experience and causal properties of physical systems: The conceptual structure specified by a complex of elements in a state is identical to its experience.

Specifically, the form of the conceptual structure in cause-effect space completely specifies the quality of the experience, while the irreducibility $\Phi^{Max}$ of the conceptual structure specifies the level to which it exists (i.e., the complex's level of consciousness). The maximally irreducible cause-effect repertoire of each concept within a conceptual structure specifies what the concept contributes to the quality of the experience, while its irreducibility $\varphi^{Max}$ specifies how much the concept is present in the experience.

According to IIT, an experience is thus an intrinsic property of a complex of mechanisms in a state.

Extensions

The calculation of even a modestly-sized system's $\Phi^{Max}$ is often computationally intractable, so efforts have been made to develop heuristic or proxy measures of integrated information. For example, Masafumi Oizumi has developed $\Phi^*$, a practical approximation for integrated information that solves the theoretical shortcomings of previously proposed proxy measures, such as the one proposed by Adam Barrett.

A significant computational challenge in calculating integrated information is finding the Minimum Information Partition of a neural system, which requires iterating through all possible network partitions. To solve this problem, Daniel Toker has suggested using the most modular decomposition of a network as an extremely quick proxy for the Minimum Information Partition.

Related Experimental Work

While the algorithm for assessing a system's $\Phi^{Max}$ and conceptual structure is relatively straightforward, its high time complexity makes it computationally intractable for many systems of interest. Heuristics and approximations can sometimes be used to provide ballpark estimates of a complex system's integrated information, but precise calculations are often impossible. These computational challenges, combined with the already difficult task of reliably and accurately assessing consciousness under experimental conditions, make testing many of the theory's predictions difficult.

Despite these challenges, researchers have attempted to use measures of information integration and differentiation to asses levels of consciousness in a variety of subjects. For instance, a recent study using a less computationally-intensive proxy for $\Phi^{Max}$ was able to reliably discriminate between varying levels of consciousness in wakeful, sleeping (dreaming vs. non-dreaming), anesthetized, and comatose (vegetative vs. minimally-conscious vs. locked-in) individuals.

IIT also makes several predictions which fit well with existing experimental evidence, and can be used to explain some counterintuitive findings in consciousness research. For example, IIT can be used to explain why some brain regions, such as the cerebellum do not appear to contribute to consciousness, despite their size and/or functional importance. IIT can also help to explain why severing the corpus callosum appears to lead to the development of two separate consciousnesses in split-brain patients.

Reception

Integrated Information Theory has received both broad criticism and support.

Support

Neuroscientist Christof Koch has called IIT "the only really promising fundamental theory of consciousness.”

Criticism

Meanwhile, some critics have challenged that IIT proposes conditions which are necessary for consciousness, but are not entirely sufficient. Objections have also been made to the claim that all of IIT's axioms are self-evident. Since IIT is not a functionalist theory of consciousness, many historical criticisms of non-functionalism have been applied to IIT. Disagreements over the definition of consciousness also lead to inevitable criticism of the theory.

See also

• Consciousness
• Hard problem of consciousness
• Mind–body problem
• Phenomenology (psychology)
• Phenomenology (philosophy)
• Qualia
• Causality
• Sentience
• Philosophy of mind

External links

Related papers

• From the Phenomenology to the Mechanisms of Consciousness: Integrated Information Theory 3.0
• Integrated Information Theory: An Updated Account (2012) (First presentation of IIT 3.0)
• Integrated Information Theory: A Provisional Manifesto (2008) (IIT 2.0)
• An Information Integration Theory of Consciousness (2004) (IIT 1.0)

References

• Integrated information theory (Scholarpedia article)

Websites

• IntegratedInformationTheory.org - Maintains software for calculating IIT quantities.

Books

• Phi: A Voyage from the Brain to the Soul

News articles

• MIT Technology Review (2014): What It Will Take for Computers to Be Conscious
• Wired (2013): A Neuroscientist’s Radical Theory of How Networks Become Conscious
• The New Yorker (2013): How Much Consciousness Does an iPhone Have?
• New York Times (2010): Sizing Up Consciousness by Its Bits
• Scientific American (2009): A “Complex” Theory of Consciousness
• IEEE Spectrum (2008): A Bit of Theory: Consciousness as Integrated Information Theory

Talks

• David Chalmers (2014): How do you explain consciousness?
• Christof Koch (2014): The Integrated Information Theory of Consciousness