Duplicate characters in Unicode
From Wikipedia, the free encyclopedia
Unicode has a certain amount of duplication due to aiming to allow legacy encodings to be converted to Unicode without losing any information. Sometimes these characters are rendered identically, other times they are rendered with different sizes or styles at least in the fonts intended to match up with the expectations of legacy systems.
Contents |
[edit] Weaknesses of duplicate characters
The similarity of some Unicode characters can be used as an attack vector in Phishing scams. Standard domain names for internet sites allow a limited Latin character set, but the Punycode standard has recently enabled Internationalized domain names in all languages supported by Unicode. This makes it possible to register a domain name with homographs for a legitimate site. For more information, see IDN homograph attack.
[edit] CJK fullwidth forms
In traditional CJK encodings characters usually took either a single byte (known as halfwidth) or two bytes (known as fullwidth). Characters that took a single byte were generally displayed at half the width of those that took two bytes. Some characters such as the latin alphabet were available in both halfwidth and fullwidth versions. As the halfwidth versions were more commonly used they were generally the ones mapped to the standard code points for those characters. Therefore a separate section was needed for the fullwidth forms to preserve the distinction.
[edit] Greek
Many Greek letters are used as technical symbols. All of the Greek letters are encoded in the Greek section of Unicode but many are encoded a second time under the name of the technical symbol they represent. Of these, micro sign is in the Latin-1 range and most of the rest are in the Letterlike Symbols range. The "micro sign" (U+00B5, µ) is obviously inherited from ISO 8859-1, but the origin of the others is less clear.
Technically these are not duplicate characters in that the consortium viewed these symbols used in mathematics as distinct characters from their twins in the Greek alphabet. For example, the upper-case character pi (U+03A0, Π) typically shares the same glyph with the mathematical character n-ary product (U+220F, ∏). These are two different characters because they each represent different semantics — different meanings. They nevertheless share similar presentational form: the glyph. Humans can typically perceive the difference in characters through the context in which the glyph appears. On the other hand, it is difficult to develop software that can distinguish these different meanings through context: hence the justification for providing different characters for these different meanings.
[edit] Roman numerals
Unicode has a number of characters specifically designated as Roman numerals, as part of the Number Forms range from U+2160 to U+2183. For example, MCMLXXXVIII could alternatively be written as ⅯⅭⅯⅬⅩⅩⅩⅧ. This range includes both upper- and lowercase numerals, as well as pre-combined glyphs for numbers up to 12 (Ⅻ or XII), mainly intended for the clock faces for compatibility with non–West-European encodings. The pre-combined glyphs should only be used to represent the individual numbers where the use of individual glyphs is not wanted, and not to replace compounded numbers. Similarly precombined ligatures for 5000 and 10000 exist.
| uppercase one | U+2160 | Ⅰ |
| uppercase two | U+2161 | Ⅱ |
| uppercase three | U+2162 | Ⅲ |
| uppercase four | U+2163 | Ⅳ |
| uppercase five | U+2164 | Ⅴ |
| uppercase six | U+2165 | Ⅵ |
| uppercase seven | U+2166 | Ⅶ |
| uppercase eight | U+2167 | Ⅷ |
| uppercase nine | U+2168 | Ⅸ |
| uppercase ten | U+2169 | Ⅹ |
| uppercase eleven | U+216A | Ⅺ |
| uppercase twelve | U+216B | Ⅻ |
| uppercase fifty | U+216C | Ⅼ |
| uppercase one hundred | U+216D | Ⅽ |
| uppercase five hundred | U+216E | Ⅾ |
| uppercase one thousand | U+216F | Ⅿ |
| lowercase one | U+2170 | ⅰ |
| lowercase two | U+2171 | ⅱ |
| lowercase three | U+2172 | ⅲ |
| lowercase four | U+2173 | ⅳ |
| lowercase five | U+2174 | ⅴ |
| lowercase six | U+2175 | ⅵ |
| lowercase seven | U+2176 | ⅶ |
| lowercase eight | U+2177 | ⅷ |
| lowercase nine | U+2178 | ⅸ |
| lowercase ten | U+2179 | ⅹ |
| lowercase eleven | U+217A | ⅺ |
| lowercase twelve | U+217B | ⅻ |
| lowercase fifty | U+217C | ⅼ |
| lowercase one hundred | U+217D | ⅽ |
| lowercase five hundred | U+217E | ⅾ |
| lowercase one thousand | U+217F | ⅿ |
| one thousand C D | U+2180 | ↀ |
| five thousand | U+2181 | ↁ |
| ten thousand | U+2182 | ↂ |
| reverse one hundred | U+2183 | Ↄ |
Some of these characters are considered duplicates by Unicode methodology because many of these characters can be composed from the other characters. For example, one can combine Ⅹ with Ⅰ to mean roman numeral eleven (ⅩⅠ), so U+216A (Ⅺ) is a pre-composed character. Such characters are also referred to as composite compatibility characters or decomposable compatibility characters. Such characters would not normally have been included within the Unicode standard except for compatibility with other existing encodings. The goal was to accommodate simple translation from existing encodings into unicode. This makes translations in the opposite direction complicated because multiple Unicode characters may map to a single character in another encoding. Without the compatibility concerns the only characters necessary would be: Ⅰ, Ⅴ, Ⅹ, Ⅼ, Ⅽ, Ⅾ, Ⅿ, ⅰ, ⅴ, ⅹ, ⅼ, ⅽ, ⅾ, ⅿ, ↀ, ↁ, ↂ, Ↄ. All other roman numerals can be composed from these.fr:Duplication de caractères unicode
