Context-sensitive grammar

A context-sensitive grammar (CSG) is a formal grammar in which the left-hand sides and right-hand sides of any production rules may be surrounded by a context of terminal and nonterminal symbols. Context-sensitive grammars are more general than context-free grammars, in the sense that there are some languages that cannot be described by context-free grammars, but can be described by CSG. Context-sensitive grammars are however less general (in the same sense of the term) than unrestricted grammars, i.e. CSG occupy the intermediate position between context-free and unrestricted grammars in the Chomsky hierarchy.

A formal language that can be described by a context-sensitive grammar, or, equivalently, by a noncontracting grammar or a linear bounded automaton, is called a context-sensitive language. Some textbooks actually define CSG as non-contracting, although this is not how Noam Chomsky defined it in 1959. This choice of definition makes no difference in terms of the languages generated (i.e. the two definitions are weakly equivalent), but it does make a difference in terms of what grammars are structurally considered context-sensitive; the later issue was analyzed by Chomsky in 1963.

Chomsky introduced context-sensitive grammars as a way to describe the syntax of natural language where it is indeed often the case that a word may or may not be appropriate in a certain place depending upon the context. Walter Savitch has criticized the terminology "context-sensitive" as misleading and proposed "non-erasing" as better explaining the distinction between a CSG and an unrestricted grammar.

Although it is well-known that certain features of languages (e.g. cross-serial dependency) are not context-free, it is an open research question how much of CSG' expressive power is actually needed to capture the context sensitivity found in natural languages. Subsequent research in this area has focused on the more computationally tractable mildly context-sensitive languages.

Formal definition

A formal grammar G = (N, Σ, P, S), where N is a set of nonterminal symbols, Σ is a set of terminal symbols, P is a set of production rules, and S is the start symbol, is context-sensitive if all rules in P are of the form

αAβ → αγβ

where A ∈ N, α,β ∈ (N∪Σ)^* and γ ∈ (N∪Σ)⁺.

A string u ∈ (N∪Σ)^* directly yields, or directly derives to, a string v ∈ (N∪Σ)^*, denoted as u ⇒ v, if u can be written as lαAβr, and v can be written as lαγβr, for some production rule (αAβ→αγβ) ∈ P, and some context strings l, r ∈ (N∪Σ)^*. More generally, u is said to yield, or derive to, v, denoted as u ⇒^* v, if u = u₁ ⇒ ... ⇒ u_n = v for some n≥0 and some strings u₂, ..., u_n-1 (N∪Σ)^*. That is, the relation (⇒^*) is the reflexive transitive closure of the relation (⇒).

The language of the grammar G is the set of all terminal symbol strings derivable from its start symbol, formally: L(G) = { w ∈ Σ^*: S ⇒^* w }. Derivations that do not end in a string composed of terminal symbols only are possible, but don't contribute to L(G).

The only difference between this definition of Chomsky and that of unrestricted grammars is that γ can be empty in the unrestricted case.

Some definitions of a context-sensitive grammar only require that for any production rule of the form u → v, the length of u shall be less than or equal to the length of v. This seemingly weaker requirement is in fact weakly equivalent, see Noncontracting grammar#Transforming into context-sensitive grammar.

In addition, a rule of the form

S → λ

where λ represents the empty string and S does not appear on the right-hand side of any rule is permitted. The addition of the empty string allows the statement that the context sensitive languages are a proper superset of the context free languages, rather than having to make the weaker statement that all context free grammars with no →λ productions are also context sensitive grammars.

The name context-sensitive is explained by the α and β that form the context of A and determine whether A can be replaced with γ or not. This is different from a context-free grammar where the context of a nonterminal is not taken into consideration. Indeed, every production of a context free grammar is of the form V → w where V is a single nonterminal symbol, and w is a string of terminals and/or nonterminals; w can be empty.

If the possibility of adding the empty string to a language is added to the strings recognized by the noncontracting grammars (which can never include the empty string) then the languages in these two definitions are identical.

The left-context- and right-context-sensitive grammars are defined by restricting the rules to just the form αA → αγ and to just Aβ → γβ, respectively. The languages generated by these grammars are also the full class of context-sensitive languages. The equivalence was established by Penttonen normal form.

Examples

The following grammar, with start symbol S, generates the canonical non-context-free language { aⁿbⁿcⁿ : n ≥ 1 } :

1.		S	→	a	b	c
2.		S	→	a	S	B	c
3.	c	B	→	W	B
4.	W	B	→	W	X
5.	W	X	→	B	X
6.	B	X	→	B	c
7.	b	B	→	b	b

Rules 1 and 2 allow for blowing-up S to aⁿbc(Bc)^n-1; rules 3 to 6 allow for successively exchanging each cB to Bc (four rules are needed for that since a rule cB → Bc wouldn't fit into the scheme αAβ → αγβ); rule 7 allows for replacing a non-terminal B with its corresponding terminal b, provided it is in the right place. A generation chain for aaabbbccc is:

S

→₂ aSBc

→₂ aaSBcBc

→₁ aaabcBcBc

→₃ aaabWBcBc

→₄ aaabWXcBc

→₅ aaabBXcBc

→₆ aaabBccBc

→₃ aaabBcWBc

→₄ aaabBcWXc

→₅ aaabBcBXc

→₆ aaabBcBcc

→₃ aaabBWBcc

→₄ aaabBWXcc

→₅ aaabBBXcc

→₆ aaabBBccc

→₇ aaabbBccc

→₇ aaabbbccc

More complicated grammars can be used to parse { aⁿbⁿcⁿdⁿ: n ≥ 1 }, and other languages with even more letters.

A context-sensitive grammar for the language { a²ⁱ : i ≥ 1 } is constructed in Example 9.5 (p. 224) of (Hopcroft, Ullman, 1979).

Kuroda normal form

Every context-sensitive grammar which does not generate the empty string can be transformed into a weakly equivalent one in Kuroda normal form. "Weakly equivalent" here means that the two grammars generate the same language. The normal form will not in general be context-sensitive, but will be a noncontracting grammar.

The Kuroda normal form is an actual normal form for non-contracting grammars.

Properties and uses

See also: context-sensitive language

Equivalence to linear bounded automaton

A formal language can be described by a context-sensitive grammar if and only if it is accepted by some linear bounded automaton (LBA). In some textbooks this result is attributed solely to Landweber and Kuroda. Others call it the Myhill-Landweber-Kuroda Theorem. (Myhill introduced the concept of deterministic LBA in 1960. Peter S. Landweber published in 1963 that the language accepted by a deterministic LBA is context sensitive. Kuroda introduced the notion of non-deterministic LBA and the equivalence between LBA and CSGs in 1964.)

As of 2010 it is still an open question whether every context-sensitive language can be accepted by a deterministic LBA.

Closure properties

Context-sensitive languages are closed under complement. This 1988 result is known as the Immerman–Szelepcsényi theorem. Moreover, they are closed under union, intersection, concatenation, substitution, inverse homomorphism, and Kleene plus.

Every recursively enumerable language L can be written as h(L) for some context-sensitive language L and some string homomorphism h.

Computational problems

The decision problem that asks whether a certain string s belongs to the language of a given context-sensitive grammar G, is PSPACE-complete. Morever, there are context-sensitive grammars whose languages are PSPACE-complete. In other words, there is a context-sensitive grammar G such that deciding whether a certain string s belongs to the language of G is PSPACE-complete (so G is fixed and only s is part of the input of the problem).

The emptiness problem for context-sensitive grammars (given a context-sensitive grammar G, is L(G)=∅ ?) is undecidable.

As model of natural languages

Savitch has proven the following theoretical result, on which he bases his criticism of CSGs as basis for natural language: for any recursively enumerable set R, there exists a context-sensitive language/grammar G which can be used as a sort of proxy to test membership in R in the following way: given a string s, s is in R if and only if there exists a positive integer n for which scⁿ is in G, where c is an arbitrary symbol not part of R.

It has been shown that nearly all natural languages may in general be characterized by context-sensitive grammars, but the whole class of CSG's seems to be much bigger than natural languages. Worse yet, since the aforementioned decision problem for CSG's is PSPACE-complete, that makes them totally unworkable for practical use, as a polynomial-time algorithm for a PSPACE-complete problem would imply P=NP.

It was proven that some natural languages are not context-free, based on identifying so-called cross-serial dependencies and unbounded scrambling phenomena. However this does not necessarily imply that all the class CSG is necessary to capture "context sensitivity" in the colloquial sense of these terms in natural languages. For example, the strictly weaker (than CSG) linear context-free rewriting systems (LCFRS) can account for the phenomenon of cross-serial dependencies; one can write a LCFRS grammar for {aⁿbⁿcⁿdⁿ | n ≥ 1} for example.

Ongoing research on computational linguistics has focused on formulating other classes of languages that are "mildly context-sensitive" whose decision problems are feasible, such as tree-adjoining grammars, combinatory categorial grammars, coupled context-free languages, and linear context-free rewriting systems. The languages generated by these formalisms properly lie between the context-free and context-sensitive languages.

More recently, the class PTIME has been identified with range concatenation grammars, which are now considered to be the most expressive of the mild-context sensitive languages.

See also

• Chomsky hierarchy
• Growing context-sensitive grammar

External links

• Earley Parsing for Context-Sensitive Grammars