ISO/IEC 2022

From Infogalactic: the planetary knowledge core
(Redirected from ISO 2022)
Jump to: navigation, search

<templatestyles src="Module:Hatnote/styles.css"></templatestyles>

Not to be confused with ISO 20022.
ISO 2022
Standard ISO/IEC 2022,
ECMA-35,
JIS X 0202
Language(s) Various.
Succeeded by ISO 10646 (Unicode)

ISO/IEC 2022 Information technology—Character code structure and extension techniques, is an ISO standard (equivalent to the ECMA standard ECMA-35[1]) specifying

  • a technique for including multiple character sets in a single character encoding system, and
  • a technique for representing these character sets in both 7 and 8 bit systems using the same encoding.

Many of the character sets included as ISO/IEC 2022 encodings are 'double byte' encodings where two bytes correspond to a single character. This makes ISO-2022 a variable width encoding. But a specific implementation does not have to implement all of the standard; the conformance level and the supported character sets are defined by the implementation.

Introduction

Lua error in package.lua at line 80: module 'strict' not found. Many languages or language families not based on the Latin alphabet such as Greek, Cyrillic, Arabic, or Hebrew have historically been represented on computers with different 8-bit extended ASCII encodings. Written East Asian languages, specifically Chinese, Japanese, and Korean, use far more characters than can be represented in an 8-bit computer byte and were first represented on computers with language-specific double byte encodings.

ISO/IEC 2022 was developed as a technique to attack both of these problems: to represent characters in multiple character sets within a single character encoding, and to represent large character sets.

A second requirement of ISO-2022 was that it should be compatible with 7-bit communication channels. So even though ISO-2022 is an 8-bit character set any 8-bit sequence can be reencoded to use only 7-bits without loss and normally only a small increase in size.

To represent multiple character sets, the ISO/IEC 2022 character encodings include escape sequences which indicate the character set for characters which follow. The escape sequences are registered with ISO and follow the patterns defined within the standard. These character encodings require data to be processed sequentially in a forward direction since the correct interpretation of the data depends on previously encountered escape sequences. Note, however, that other standards such as ISO-2022-JP may impose extra conditions such as the current character set is reset to US-ASCII before the end of a line.

To represent large character sets, ISO/IEC 2022 builds on ISO/IEC 646's property that one seven bit character will normally define 94 graphic (printable) characters (in addition to space and 33 control characters). Using two bytes, it is thus possible to represent up to 8836 (94×94) characters; and, using three bytes, up to 830584 (94×94×94) characters. Though the standard defines it, no registered character set uses three bytes (although EUC-TW's unregistered G2 is). For the two-byte character sets, the code point of each character is normally specified in so-called kuten (Japanese: 区点) form (sometimes called qūwèi (Chinese: 区位), especially when dealing with GB2312 and related standards), which specifies a zone (区, Japanese: ku, Chinese: qū), and the point (Japanese: 点 ten) or position (Chinese: 位 wèi) of that character within the zone.

The escape sequences therefore do not only declare which character set is being used, but also, by knowing the properties of these character sets, know whether a 94-, 96-, 8836-, or 830584-character (or some other sized) encoding is being dealt with.

In practice, the escape sequences declaring the national character sets may be absent if context or convention dictates that a certain national character set is to be used. For example, ISO-8859-1 states that no defining escape sequence is needed and RFC 1922, which defines ISO-2022-CN, allows ISO-2022 SHIFT characters to be used without explicit use of escape sequences.

The ISO-2022 definitions of the ISO-8859-X character sets are specific fixed combinations of the components that form ISO-2022. Specifically the lower control characters (C0) the US-ASCII character set (in GL) and the upper control characters (C1) are standard and the high characters (GR) are defined for each of the ISO-8859-X variants; for example ISO-8859-1 is defined[citation needed] by the combination of ISO-IR-1, ISO-IR-6, ISO-IR-77 and ISO-IR-100 with no shifts or character changes allowed.

Although ISO/IEC 2022 character sets using control sequences are still in common use, particularly ISO-2022-JP, most modern e-mail applications are converting to use the simpler Unicode transforms such as UTF-8. The encodings that don't use control sequences, such as the ISO-8859 sets are still very common.

Code structure

Notation and nomenclature

ISO/IEC 2022 coding specifies a two-layer mapping between character codes and displayed characters. Escape sequences allow any of a large registry of graphic character sets to be "designated"[2] into one of four working sets, named G0 through G3, and shorter control sequences specify the working set that is "invoked"[3] to interpret bytes in the stream.

Encoding byte values ("bit combinations") are often given in column-line notation, where two decimal numbers in the range 00–15 (each corresponding to a single hexadecimal digit) are separated by a slash.[4] Hence also, codes 0x20 through 0x2F inclusive may be referred to as "column 02". This is the notation used in the ISO/IEC 2022 / ECMA-35 standard itself.[5] They may be described elsewhere using hexadecimal, as is often used in this article, or using the corresponding ASCII characters,[6] although the escape sequences are actually defined in terms of byte values, and the graphic assigned to that byte value may be altered without affecting the control sequence.

Byte values from the 7-bit ASCII graphic range (hexadecimal 0x20–0x7F), being on the left side of a character code table, are referred to as "GL" codes (with "GL" standing for "graphics left") while bytes from the "high ASCII" range (0xA0–0xFF), if available (i.e. in an 8-bit environment), are referred to as the "GR" codes ("graphics right").[7] The terms "CL" and "CR" are defined for the control ranges, but the CL range always invokes the primary (C0) controls, whereas the CR range always either invokes the secondary (C1) controls or is unused.[7]

Fixed coded characters

The delete character DEL (0x7F), the escape character ESC (0x1B) and the space character SP (0x20) are designated "fixed" coded characters[8] and are always available when G0 is invoked over GL, irrespective of what character sets are designated. They may not be included in graphical character sets, although other sizes or types of whitespace character may be.[9]

Graphical character sets

Each of the four working sets G0 through G3 may be a 94-character set or a 94n-character multi-byte set. Additionally, G1 through G3 may be a 96- or 96n-character set.

In a 96- or 96n-character set, the bytes 0x20 through 0x7F (or 0xA0 through 0xFF) are used. In a 94- or 94n-character set, the bytes 0x20 and 0x7F are not used.[10] When a 96- or 96n-character set is invoked in the GL region, the space and delete characters (codes 0x20 and 0x7F) are not available until a 94- or 94n-character set (such as the G0 set) is invoked in GL.[7]

Combining characters

Characters are expected to be spacing characters, not combining characters, unless specified otherwise by the graphical set in question.[11] ISO 2022 / ECMA-35 also recognises the use of the backspace and carriage return control characters as means of combining otherwise spacing characters, as well as the CSI sequence "Graphic Character Combination" (GCC)[11] (CSI 0x20 (SP) 0x5F (_)).[12]

Use of the backspace and carriage return in this manner is permitted by ISO/IEC 646 but prohibited by ISO/IEC 4873 / ECMA-43[13] and by ISO/IEC 8859,[14][15] on the basis that it leaves the graphical character repertoire undefined.[13] ISO/IEC 4873 / ECMA-43 does, however, permit the use of the GCC function on the basis that the sequence of characters is kept the same and merely displayed in one space, rather than being over-stamped to form a character with a different meaning.[13]

Control character sets

Control character sets are classified as "primary" or "secondary" control character sets,[16] respectively also called "C0" and "C1" control character sets.[17]

A C0 control set must contain the ESC (escape) control character at 0x1B,[18] whereas a C1 control set may not contain the escape control whatsoever.[19] Hence, they are entirely separate registrations, with a C0 set being only a C0 set and a C1 set being only a C1 set.[17] If codes from the C0 set of ISO 6429 / ECMA-48, i.e. the ASCII control codes, appear in the C0 set, they are required to appear at their ISO 6429 / ECMA-48 locations.[18] Inclusion of transmission control characters in the C0 set, besides the ten included by ISO 6429 / ECMA-48, or inclusion of any of those ten in the C1 set, is also prohibited by the ISO/IEC 2022 / ECMA-35 standard.[18][19]

A C0 control set is invoked over the CL range 0x00 through 0x1F,[20] whereas a C1 control character may be invoked over the CR range 0x80 through 0x9F (in an 8-bit environment) or by using escape sequences (in a 7-bit or 8-bit environment),[16] but not both. Which style of C1 invocation is used must be specified in the definition of the code version.[21] For example, ISO/IEC 4873 specifies CR bytes for the C1 controls (SS2 and SS3) which it uses.[13]

In the latter case, single control characters from the C1 control character set are invoked using "type Fe" escape sequences,[19] meaning those where the ESC control character is followed by a byte from columns 04 or 05 (that is to say, ESC 0x40 (@) through ESC 0x5F (_)).[22]

Other control functions

Additional control functions are assigned to "type Fs" escape sequences (in the range ESC 0x60 (`) through ESC 0x7E (~)); these have permanently assigned meanings rather than depending on the C0 or C1 designations.[22][23] Registration of control functions to type "Fs" sequences must be approved by ISO/IEC JTC 1/SC 2.[23] Other single control functions are registered to "type 3Ft" escape sequences (in the range ESC 0x23 (#) 0x40 (@) through ESC 0x23 (#) 0x7E (~)).[24]

Escape sequences of type "Fp" (ESC 0x30 (0) through ESC 0x3F (?)) or of type "3Fp" (ESC 0x23 (#) 0x30 (0) through ESC 0x23 (#) 0x3F (?)) are reserved for private use, by prior agreement between parties.[25]

Shift functions

By default, GL codes specify G0 characters, and GR codes specify G1 characters, but this may be modified with control codes or by prior agreement:[26]

Code Hex Abbr. Name Effect
SI 0F SI
LS0		 Shift In
Locking shift zero		 GL encodes G0 from now on[27]
SO 0E SO
LS1		 Shift Out
Locking shift one		 GL encodes G1 from now on[27]
ESC n 1B 6E LS2 Locking shift two GL encodes G2 from now on[27]
ESC o 1B 6F LS3 Locking shift three GL encodes G3 from now on[27]
CR area: SS2
Escape code: ESC N		 CR area: 8E
Escape code: 1B 4E		 SS2 Single shift two GL or GR (see below) encodes G2 for the immediately following character only
CR area: SS3
Escape code: ESC O		 CR area: 8F
Escape code: 1B 4F		 SS3 Single shift three GL or GR (see below) encodes G3 for the immediately following character only
ESC ~ 1B 7E LS1R Locking shift one right GR encodes G1 from now on[28]
ESC } 1B 7D LS2R Locking shift two right GR encodes G2 from now on[28]
ESC | 1B 7C LS3R Locking shift three right GR encodes G3 from now on[28]

In 8-bit environments, either GL or GR, but not both, may be used as the single-shift area. This must be specified in the definition of the code version.[29] For instance, ISO/IEC 4873 specifies GL, whereas packed EUC specifies GR. In 7-bit environments, only GL is used as the single-shift area.[30]

Since SI (LS0) and SO (LS1) are C0 controls and SS2 and SS3 are C1 controls, they must be present in the respective sets if their functionality is used.[20][21] The C1 controls themselves, as mentioned above, may be represented using escape sequences or 8-bit bytes, but not both.

The ISO/IEC 2022 / ECMA-35 standard permits, but discourages, invoking G1, G2 or G3 in both GL and GR simultaneously.[31]

Character set designations

Escape sequences to designate character sets take the form ESC I [I...] F, where there are one or more intermediate bytes[32] (I) from the range 0x20–0x2F, and a final byte[33] (F) from the range 0x30–0x7F.[34] However, the range 0x30–0x3F is reserved for private-use F bytes for unregistered uses defined by prior agreement between parties,[35] which might be defined by further protocols such as ARIB STD-B24.

The I bytes identify the type of character set and the working set it is to be designated to, while the F byte identifies the character set itself. Additional I bytes may be added before the F byte to extend the F byte range. This is currently only used with 94-character sets, where codes of the form ESC ( ! F have been assigned. At the other extreme, no multibyte 96-sets have been registered, so the sequences below are strictly theoretical.

There are also three special cases for multi-byte codes. The code sequences ESC $ @, ESC $ A, and ESC $ B were all registered when the contemporary version of the standard allowed multi-byte sets only in G0, so must be accepted as synonyms for the sequences ESC $ ( @ through ESC $ ( B to designate to the G0 character set.[36]

There are additional (rarely used) features for switching control character sets, but this is a single-level lookup: the 0x00–0x1F range is the C0 control character set, the 0x80–0x9F range is the C1 control character set, and there are escape sequences which switch in various alternatives. It is required that any C0 character set include the ESC character at position 0x1B, so that further changes are possible.

The standard also defines a way to specify coding systems that do not follow its own structure. Of particular interest: the sequence ESC % G switches to the UTF-8 coding system (which does not reserve the range 0x80–0x9F for control characters),[37][38] the sequence ESC % @ switches back to ISO 2022,[39] and the sequence ESC % / I switches to UTF-8 permanently.[37][40]

Code Hex Abbr. Name Effect Example
ESC ! F 1B 21 F CZD C0-designate F selects a C0 control character set to be used.[41] ESC ! @
(ASCII C0 codes)
ESC " F 1B 22 F C1D C1-designate F selects a C1 control character set to be used.[42] ESC " C
(ISO 6429 C1 codes)
ESC % F 1B 25 F DOCS Designate other coding system F selects an 8-bit code; use ESC % @ to return to ISO/IEC 2022.[39] ESC % G
(UTF-8)[37][38][lower-alpha 1]
ESC % / F 1B 25 2F F DOCS Designate other coding system F selects an 8-bit code; there is no standard way to return.[lower-alpha 2] ESC % / L
(UTF-16)[37][44][lower-alpha 3]
ESC & F 1B 26 F IRR Identify revised registration Prefixes designation escape to denote revision.[lower-alpha 4] ESC & @ ESC $ B
(JIS X 0208:1990 in G0)
ESC ( F 1B 28 F GZD4 G0-designate 94-set F selects a 94-character set to be used for G0.[36] ESC ( B
(ASCII in G0)
ESC ) F 1B 29 F G1D4 G1-designate 94-set F selects a 94-character set to be used for G1.[36] ESC ) I
(JIS X 0201 Kana in G1)
ESC * F 1B 2A F G2D4 G2-designate 94-set F selects a 94-character set to be used for G2.[36] ESC * v
(ITU T.61 RHS in G2)
ESC + F 1B 2B F G3D4 G3-designate 94-set F selects a 94-character set to be used for G3.[36] ESC + D
(NATS-SEFI-ADD in G3)
ESC - F 1B 2D F G1D6 G1-designate 96-set F selects a 96-character set to be used for G1.[36] ESC - A
(ISO 8859-1 RHS in G1)
ESC . F 1B 2E F G2D6 G2-designate 96-set F selects a 96-character set to be used for G2.[36] ESC . B
(ISO 8859-2 RHS in G2)
ESC / F 1B 2F F G3D6 G3-designate 96-set F selects a 96-character set to be used for G3.[36] ESC / b
(ISO 8859-15 RHS in G3)
ESC $ F[lower-alpha 5]
ESC $ ( F		 1B 24 F[lower-alpha 5]
1B 24 28 F		 GZDM4 G0-designate multibyte 94-set F selects a 94n-character set to be used for G0.[36] ESC $ ( C
(KS X 1001 in G0)
ESC $ ) F 1B 24 29 F G1DM4 G1-designate multibyte 94-set F selects a 94n-character set to be used for G1.[36] ESC $ ) A
(GB 2312 in G1)
ESC $ * F 1B 24 2A F G2DM4 G2-designate multibyte 94-set F selects a 94n-character set to be used for G2.[36] ESC $ * B
(JIS X 0208 in G2)
ESC $ + F 1B 24 2B F G3DM4 G3-designate multibyte 94-set F selects a 94n-character set to be used for G3.[36] ESC $ + D
(JIS X 0212 in G3)
ESC $ - F 1B 24 2D F G1DM6 G1-designate multibyte 96-set F selects a 96n-character set to be used for G1.[36] ESC $ - 1
(private use)
ESC $ . F 1B 24 2E F G2DM6 G2-designate multibyte 96-set F selects a 96n-character set to be used for G2.[36] ESC $ . 2
(private use)
ESC $ / F 1B 24 2F F G3DM6 G3-designate multibyte 96-set F selects a 96n-character set to be used for G3.[36] ESC $ + 3
(private use)

Note that the registry of F bytes is independent for the different types. The 94-character graphic set designated by ESC ( A through ESC + A is not related in any way to the 96-character set designated by ESC - A through ESC / A. And neither of those is related to the 94n-character set designated by ESC $ ( A through ESC $ + A, and so on; the final bytes must be interpreted in context. (Indeed, without any intermediate bytes, ESC A is a way of specifying the C1 control code 0x81.)

Also note that C0 and C1 control character sets are independent; the C0 control character set designated by ESC ! A (which happens to be the NATS control set for newspaper text transmission) is not the same as the C1 control character set designated by ESC " A (the CCITT attribute control set for Videotex).

ISO/IEC 2022 code versions

Various ISO 2022 and other CJK encodings supported by Mozilla Firefox as of 2004. (This support has been reduced in later versions to avoid certain cross site scripting attacks.)

Japanese e-mail versions

ISO-2022-JP is a widely used encoding for Japanese, in particular in e-mail. It is codified in IETF RFC 1468, dated 1993.[46] It has an advantage over other encodings for Japanese in that it does not require 8-bit clean transmission. Microsoft calls it Code page 50220.[47] It starts in ASCII and includes the following escape sequences:

  • ESC ( B to switch to ASCII (1 byte per character)
  • ESC ( J to switch to JIS X 0201-1976 (ISO/IEC 646:JP) Roman set (1 byte per character)
  • ESC $ @ to switch to JIS X 0208-1978 (2 bytes per character)
  • ESC $ B to switch to JIS X 0208-1983 (2 bytes per character)

Use of ESC ( I to switch to the JIS X 0201-1976 Kana set (1 byte per character) is not part of the ISO-2022-JP profile, but is also sometimes used. Python calls this variant ISO-2022-JP-EXT,[48][49] while Microsoft calls it Code page 50221.[47] The WHATWG encoding standard referenced by HTML5 permits decoding JIS X 0201 katakana in ISO-2022-JP input, but converts the characters to their JIS X 0208 equivalents upon encoding.[50]

ISO-2022-JP-2 is a multilingual extension of ISO-2022-JP, defined in RFC 1554 (dated 1993), which permits the following escape sequences in addition to the ISO-2022-JP ones. The ISO/IEC 8859 parts are 96-character sets which cannot be designated to G0, and are accessed from G2 using the 7-bit escape sequence form of the single-shift code SS2:[51]

  • ESC $ A to switch to GB 2312-1980 (2 bytes per character)
  • ESC $ ( C to switch to KS X 1001-1992 (2 bytes per character)
  • ESC $ ( D to switch to JIS X 0212-1990 (2 bytes per character)
  • ESC . A to switch to ISO/IEC 8859-1 high part, Extended Latin 1 set (1 byte per character) [designated to G2]
  • ESC . F to switch to ISO/IEC 8859-7 high part, Basic Greek set (1 byte per character) [designated to G2]

ISO-2022-JP with the ISO-2022-JP-2 representation of JIS X 0212, but not the other extensions, was subsequently dubbed ISO-2022-JP-1 by RFC 2237, dated 1997.[52]

The JIS X 0213 standard, first published in 2000, defines an updated version of ISO-2022-JP, without the ISO-2022-JP-2 extensions, named ISO-2022-JP-3. The additions made by JIS X 0213 compared to the base JIS X 0208 standard resulted in a new registration being made for the extended JIS plane 1, while the new plane 2 received its own registration. The further additions to plane 1 in the 2004 edition of the standard resulted in an additional registration being added to a further revision of the profile, dubbed ISO-2022-JP-2004. In addition to the basic ISO-2022-JP designation codes, the following designations are recognised:

  • ESC ( I to switch to JIS X 0201-1976 Kana set (1 byte per character)
  • ESC $ ( O to switch to JIS X 0213-2000 Plane 1 (2 bytes per character)
  • ESC $ ( P to switch to JIS X 0213-2000 Plane 2 (2 bytes per character)
  • ESC $ ( Q to switch to JIS X 0213-2004 Plane 1 (2 bytes per character, ISO-2022-JP-2004 only)

Other 7-bit versions

ISO-2022-KR is defined in RFC 1557, dated 1993.[53] It encodes ASCII and the Korean double-byte KS X 1001-1992,[54][55] previously named KS C 5601-1987. Unlike ISO-2022-JP-2, it makes use of the Shift Out and Shift In characters to switch between them, after including ESC $ ) C once at the start of a line to designate KS X 1001 to G1.[53]

ISO-2022-CN and ISO-2022-CN-EXT are defined in RFC 1922, dated 1996. They are 7-bit encodings making use both of the Shift Out and Shift In functions (to shift between G0 and G1), and of the 7-bit escape code forms of the single-shift functions SS2 and SS3 (to access G2 and G3).[56] They support the character sets GB 2312 (for simplified Chinese) and CNS 11643 (for traditional Chinese).

The basic ISO-2022-CN profile includes GB 2312 and the first two planes of CNS 11643 (due to these two planes being sufficient to represent all traditional Chinese characters from common Big5, to which the RFC provides a correspondence in an appendix):[56]

  • ESC $ ) A to switch to GB 2312-1980 (2 bytes per character) [designated to G1]
  • ESC $ ) G to switch to CNS 11643-1992 Plane 1 (2 bytes per character) [designated to G1]
  • ESC $ * H to switch to CNS 11643-1992 Plane 2 (2 bytes per character) [designated to G2]

The ISO-2022-CN-EXT profile permits the following additional sets and planes.[56]

  • ESC $ ) E to switch to ISO-IR-165 (2 bytes per character) [designated to G1]
  • ESC $ + I to switch to CNS 11643-1992 Plane 3 (2 bytes per character) [designated to G3]
  • ESC $ + J to switch to CNS 11643-1992 Plane 4 (2 bytes per character) [designated to G3]
  • ESC $ + K to switch to CNS 11643-1992 Plane 5 (2 bytes per character) [designated to G3]
  • ESC $ + L to switch to CNS 11643-1992 Plane 6 (2 bytes per character) [designated to G3]
  • ESC $ + M to switch to CNS 11643-1992 Plane 7 (2 bytes per character) [designated to G3]

The ISO-2022-CN-EXT profile further lists additional Guobiao standard graphical sets as being permitted, but conditional on their being assigned registered ISO 2022 escape sequences:[56]

  • GB 12345 in G1
  • GB 7589 or GB 13131 in G2
  • GB 7590 or GB 13132 in G3

The character after the ESC (for single-byte character sets) or ESC $ (for multi-byte character sets) specifies the type of character set and working set that is designated to. In the above examples, the character ( (0x28) designates a 94-character set to the G0 character set, whereas ), * or + (0x29–0x2B) designates to the G1–G3 character sets.

ISO-2022-KR and ISO-2022-CN are used less frequently than ISO-2022-JP, and are sometimes deliberately not supported due to security concerns. Notably, the WHATWG Encoding Standard used by HTML5 maps ISO-2022-KR, ISO-2022-CN and ISO-2022-CN-EXT (as well as HZ-GB-2312) to the "replacement" decoder,[57] which maps all input to the replacement character (�), in order to prevent certain cross-site scripting and related attacks, which utilise a difference in encoding support between the client and server.[58] Although the same security concern (allowing sequences of ASCII bytes to be interpreted differently) also applies to ISO-2022-JP and UTF-16, they could not be given this treatment due to being much more frequently used in deployed content.[59]

ISO/IEC 4873

A subset of ISO 2022 applied to 8-bit single-byte encodings is defined by ISO/IEC 4873, also published by Ecma International as ECMA-43. ISO/IEC 8859 defines 8-bit codes for ISO/IEC 4873 (or ECMA-43) level 1.[14][15]

ISO/IEC 4873 / ECMA-43 defines three levels of encoding:[13]

  • Level 1, which includes a C0 set, the ASCII G0 set, an optional C1 set and an optional single-byte (94-character or 96-character) G1 set. G0 is invoked over GL, and G1 is invoked over GR. Use of shift functions is not permitted.
  • Level 2, which includes a (94-character or 96-character) single-byte G2 and/or G3 set in addition to a mandatory G1 set. Only the single-shift functions SS2 and SS3 are permitted (i.e. locking shifts are forbidden), and they invoke over the GL region (including 0x20 and 0x7F in the case of a 96-set). SS2 and SS3 must be available in C1 at 0x8E and 0x8F respectively. This minimal required C1 set for ISO 4873 is registered as ISO-IR-105.[60]
  • Level 3, which permits the GR locking-shift functions LS1R, LS2R and LS3R in addition to the single shifts, but otherwise has the same restrictions as level 2.

Earlier editions of the standard permitted non-ASCII assignments in the G0 set, provided that the ISO 646 invariant positions were preserved, that the other positions were assigned to spacing (not combining) characters, that 0x23 was assigned to either £ or #, and that 0x24 was assigned to either $ or ¤.[61] For instance, the 8-bit encoding of JIS X 0201 is compliant with earlier editions. This was subsequently changed to fully specify the ISO 646:1991 IRV / ISO-IR No. 6 set (ASCII).[13]

In cases where duplicate characters are available in different sets, the current edition of ISO/IEC 4873 / ECMA-43 only permits using these characters in the lowest numbered set which they appear in, although use in other sets is noted as having been permitted in earlier editions.[13] Use of multiple ISO/IEC 8859 parts together in ISO 2022 is prohibited by the ISO/IEC 8859 standard, which stipulates that equivalent character sets from ISO/IEC 10367 should be used at levels 2 and 3 instead.[14][15]

Character set designation escape sequences are used for identifying or switching between versions during information interchange only if required by a further protocol, in which case the standard requires an "announcer" sequence specifying the level, followed by a complete set of escapes specifying the character set designations for C0, C1, G0, G1, G2 and G3 respectively (but omitting G2 and G3 designations for level 1), with an F-byte of 0x7E denoting an empty set. Announcer sequences are as follows:[13]

Code Hex Announcement
ESC SP L 1B 20 4C ISO 4873 Level 1
ESC SP M 1B 20 4D ISO 4873 Level 2
ESC SP N 1B 20 4E ISO 4873 Level 3

Extended Unix Code

<templatestyles src="Module:Hatnote/styles.css"></templatestyles>

Main article: Extended Unix Code

Extended Unix Code (EUC) is a 8-bit variable-width character encoding system used primarily for Japanese, Korean, and simplified Chinese. It is based on ISO 2022, and only character sets which conform to the ISO 2022 structure can have EUC forms. Up to four coded character sets can be represented (in G0, G1, G2 and G3). The G0 set is invoked over GL, the G1 set is invoked over GR, and the G2 and G3 sets are (if present) invoked using the single shifts SS2 and SS3, which are used over GR (not GL). Other shift codes are not used.

Typically, G0 is used for an ISO-646 compliant single-byte coded character set such as ASCII, whereas G1 is used for a 94x94 coded character set represented in two bytes. The EUC-CN form of GB2312 and EUC-KR are examples of such two-byte EUC codes. EUC-JP includes characters represented by up to three bytes (i.e. SS3 plus two bytes) whereas a single character in EUC-TW can take up to four bytes (i.e. SS2 plus three bytes). Sometimes, ISO 646:KR (KS X 1003) or ISO 646:JP (the lower half of JIS X 0201) are used instead of ASCII for G0, meaning that 0x5C (backslash in US-ASCII) is often used to represent a Yen sign in EUC-JP and a Won sign in EUC-KR.

Comparison with other encodings

Advantages

  • As ISO/IEC 2022's entire range of graphical character encodings can be invoked over GL, the available glyphs are not significantly limited by an inability to represent GR and C1, such as in a system limited to 7-bit encodings. It accordingly enables the representation of large set of characters in such a system. Generally, this 7-bit compatibility is not really an advantage, except for backwards compatibility with older systems. The vast majority of modern computers use 8 bits for each byte.
  • As compared to Unicode, ISO/IEC 2022 sidesteps Han unification by using sequence codes to switch between discrete encodings for different East Asian languages. This avoids the issues[citation needed] associated with unification, such as difficulty supporting multiple CJK languages with their associated character variants in a single document and font.

Disadvantages

  • Since ISO/IEC 2022 is a stateful encoding, a program cannot jump in the middle of a block of text to search, insert or delete characters. This makes manipulation of the text very cumbersome and slow when compared to non-stateful encodings. Any jump in the middle of the text may require a back up to the previous escape sequence before the bytes following the escape sequence can be interpreted.
  • Due to the stateful nature of ISO/IEC 2022, an identical and equivalent character may be encoded in different character sets, which may be designated to any of G0 through G3, which may be invoked using single shifts or by using locking shifts to GL or GR. Consequently, characters can be represented in multiple ways, meaning that two visually identical and equivalent strings can not be reliably compared for equality.
  • Some systems, like DICOM and several e-mail clients, use a variant of ISO-2022 in addition to supporting several other encodings.[62] This type of variation makes it difficult to portably transfer text between computer systems.
  • UTF-1, the multi-byte Unicode transformation format compatible with ISO/IEC 2022, has various disadvantages in comparison with UTF-8, and switching from or to other charsets, as supported by ISO/IEC 2022, is typically unnecessary in Unicode documents.
  • Because of its escape sequences, it is possible to construct attack byte sequences that round-trip from ISO/IEC 2022 to Unicode and back. Use of this encoding is thus treated as suspicious by malware protection suites.[63][better source needed]
  • Concatenation can pose issues. Profiles such as ISO-2022-JP specify that the stream starts in the ASCII state and must end in the ASCII state.[64] This is necessary to ensure that characters in concatenated ISO-2022-JP and/or ASCII streams will be interpreted in the correct set. This has the consequence that if a stream that ends in a multi-byte character is concatenated with one that starts with a multi-byte character, a pair of escape codes are generated switching to ASCII and immediately away from it. However, as stipulated in Unicode Technical Report #36 ("Unicode Security Considerations"), pairs of ISO 2022 escape sequences with no characters between them should generate a replacement character ("�") to prevent them from being used to mask malicious sequences such as cross-site scripting.[65] Implementing this measure, e.g. in Mozilla Thunderbird, has led to interoperability issues, with unexpected "�" characters being generated where two ISO-2022-JP streams have been concatenated.[66]

See also

Footnotes

  1. ↑ There is also a without-standard-return sequence (ESC % / I) registered for UTF-8.[37][40][43] Additional sequences without standard return were registered for subsets of UTF-8 (ESC % / G and ESC % / H),[43] but are now deprecated.[37]
  2. ↑ A second I byte of 0x2F (/) is included in the designation sequences of codes which do not use DOCS as encoded in ISO 2022 to return to ISO 2022; it may have its own means to return to ISO 2022 or none at all.[39] All existing such registrations (as of 2019) are either transparent raw data, Unicode/UCS formats, or subsets thereof.[43]
  3. ↑ Several other without-standard-return F bytes were registered for subsets of UTF-16, namely levels 1 (J) and 2 (K) of UTF-16 and levels 1 (@), 2 (C) and 3 (E) of UCS-2.[43] They are now deprecated.[37]
  4. ↑ F, adjusted to the range 1-63, indicates which (upwardly compatible) revision of the immediately-following registration is needed, so that old systems know that they are old.[45]
  5. ↑ 5.0 5.1 For F bytes 0x40 (@), 0x41 (A) and 0x42 (B) only.[36]

References

  1. ↑ ECMA (1994), Brief History
  2. ↑ ECMA (1994), p. 4, definition 4.11
  3. ↑ ECMA (1994), p. 5, definition 4.18
  4. ↑ See, for instance, ISO-IR-14, defining the G0 designation of the JIS X 0201 Roman set as ESC 2/8 4/10.
  5. ↑ ECMA (1994), p. 5, section 5.1
  6. ↑ See, for instance, RFC 1468, defining the G0 designation of the JIS X 0201 Roman set as ESC ( J.
  7. ↑ 7.0 7.1 7.2 ECMA (1994), pp. 15–16, section 8.1
  8. ↑ ECMA (1994), p. 7, section 6.2
  9. ↑ ECMA (1994), p. 10, section 6.3.2
  10. ↑ ECMA (1994), p. 11, section 6.4.3
  11. ↑ 11.0 11.1 ECMA (1994), p. 10, section 6.3.3
  12. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  13. ↑ 13.0 13.1 13.2 13.3 13.4 13.5 13.6 13.7 Lua error in package.lua at line 80: module 'strict' not found.
  14. ↑ 14.0 14.1 14.2 Lua error in package.lua at line 80: module 'strict' not found.
  15. ↑ 15.0 15.1 15.2 Lua error in package.lua at line 80: module 'strict' not found.
  16. ↑ 16.0 16.1 ECMA (1994), p. 10, section 6.4.1
  17. ↑ 17.0 17.1 ECMA (1994), p. 11, section 6.4.4
  18. ↑ 18.0 18.1 18.2 ECMA (1994), p. 11, section 6.4.2
  19. ↑ 19.0 19.1 19.2 ECMA (1994), p. 11, section 6.4.3
  20. ↑ 20.0 20.1 ECMA (1994), p. 19, section 8.5.1
  21. ↑ 21.0 21.1 ECMA (1994), p. 19, section 8.5.2
  22. ↑ 22.0 22.1 ECMA (1994), p. 29, section 13.2.1
  23. ↑ 23.0 23.1 ECMA (1994), p. 12, section 6.5.1
  24. ↑ ECMA (1994), p. 12, section 6.5.2
  25. ↑ ECMA (1994), p. 12, section 6.5.3
  26. ↑ ECMA (1994), p. 14, table 2
  27. ↑ 27.0 27.1 27.2 27.3 ECMA (1994), p. 17, section 8.3.1
  28. ↑ 28.0 28.1 28.2 ECMA (1994), p. 17, section 8.3.2
  29. ↑ ECMA (1994), p. 19, section 8.4
  30. ↑ ECMA (1994), p. 27, section 11.1
  31. ↑ ECMA (1994), p. 17, section 8.3.3
  32. ↑ ECMA (1994), p. 4, definition 4.17
  33. ↑ ECMA (1994), p. 4, definition 4.14
  34. ↑ ECMA (1994), p. 28, section 13.1
  35. ↑ ECMA (1994), p. 33, section 13.3.3
  36. ↑ 36.00 36.01 36.02 36.03 36.04 36.05 36.06 36.07 36.08 36.09 36.10 36.11 36.12 36.13 36.14 36.15 ECMA (1994), pp. 35-36, section 14.3.2
  37. ↑ 37.0 37.1 37.2 37.3 37.4 37.5 37.6 Lua error in package.lua at line 80: module 'strict' not found.
  38. ↑ 38.0 38.1 Lua error in package.lua at line 80: module 'strict' not found.
  39. ↑ 39.0 39.1 39.2 ECMA (1994), pp. 41-42, section 15.4
  40. ↑ 40.0 40.1 Lua error in package.lua at line 80: module 'strict' not found.
  41. ↑ ECMA (1994), p. 34, section 14.2.2
  42. ↑ ECMA (1994), p. 34, section 14.2.3
  43. ↑ 43.0 43.1 43.2 43.3 Lua error in package.lua at line 80: module 'strict' not found.
  44. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  45. ↑ ECMA (1994), pp. 36-37, section 14.5
  46. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  47. ↑ 47.0 47.1 Lua error in package.lua at line 80: module 'strict' not found.
  48. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  49. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  50. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  51. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  52. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  53. ↑ 53.0 53.1 Lua error in package.lua at line 80: module 'strict' not found.
  54. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  55. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  56. ↑ 56.0 56.1 56.2 56.3 Lua error in package.lua at line 80: module 'strict' not found.
  57. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  58. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  59. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  60. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  61. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  62. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  63. ↑ https://bugzilla.mozilla.org/show_bug.cgi?id=935453
  64. ↑ RFC 1468
  65. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  66. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  • Lua error in package.lua at line 80: module 'strict' not found.
  • Lua error in package.lua at line 80: module 'strict' not found.

External links

Retrieved from "https://infogalactic.com/w/index.php?title=ISO/IEC_2022&oldid=726056082"