A Spectre is Haunting Unicode
www.dampfkraft.com
In 1978 Japan's Ministry of Economy, Trade and Industry established the encoding that would later be known as JIS X 0208, which still serves as an important reference for all Japanese encodings. However, after the JIS standard was released people noticed something strange - several of the added characters had no obvious sources, and nobody could tell what they meant or how they should be pronounced. Nobody was sure where they came from. These are what came to be known as the ghost characters (幽霊文字).
  • Ernest@lemmy.zipEnglish
    12·
    2 days ago

    The article is actually written informatively, but for anyone else who may have assumed from the title that this is some failing of Unicode–it is not, the goal of Unicode is to encode all human writing. This includes “nonsense” glyphs that have been widely used or interchanged between computer systems, including the (in)famous “RIGHT ANGLE WITH DOWNWARDS ZIGZAG ARROW” (⍼).

  • frongt@lemmy.zipEnglish
    11·
    2 days ago

    This article is underwhelming. Despite being in English, it links to a Japanese Wikipedia article. The English Wikipedia article has all the same info as this post and more: https://en.wikipedia.org/wiki/Ghost_characters

    tl;dr: some Japanese kanji from older encodings were carried forward into new ones. But some of those characters were difficult or impossible to identify. Most were archaic place names or transcription errors.

  • 🌞 Alexander Daychilde 🌞@lemmy.worldEnglish
    5·
    2 days ago

    I asked AI to tranlate it and it correctly identified the characters are ghost characters.

    I asked Google Translate which said it was simplified Chinese “The company is committed to providing high-quality products and services.”

    lol

    • brsrklf@jlai.luEnglish
      2·
      1 day ago

      It really does. Weird.

      Removing one character got me this though :

      This appears to be a string of garbled characters (mojibake) resulting from incorrect text encoding. It does not convey a meaningful message in Chinese.

      And removing even more got me to “The ancient city of Loulan”.

      • 🌞 Alexander Daychilde 🌞@lemmy.worldEnglish
        2·
        19 hours ago

        LOL. The intersection of language and machine can be fascinating.

        Two anecdotes come to mind:

        1. There’s a story out there - don’t remember the author’s name and he hasn’t written much and you used to could find the story out there but I haven’t been able to in a while as I check every so often - but “Cax6ton” will find info about it and him. It was a really great sci-fi short story (reminding me of the stories from the likes of Asimov and that timeframe) of a kid growing up and having escalating experiences of finding order in random chaos (for example, generating random letters and looking for words). I won’t spoil the ending, but it was a really nice story.

        2. My dad used to work on telephone testing equipment, and back in the 80s, there was a device they would use - DECTalk - basically you could connect it to the computer via serial cable and use a terminal to send it commands and text, and it would do text-to-speech. It was relatively primative, but the tech is in a lot of 90s-2000s+ text-to-speech that you’ll find on computers (Windows/Mac). Well, I’d spend hours writing things for it to say - oh hey, found someone has put it up on the web: https://dectalk.de/webspeak - oh, this is nostalgic!

        Some phrases that amused the SHIT out of me as a twelve year old in 1987:

        • abcdefghijklmnopqrstuvwxyz
        • taioueoiaueoiauoieuaioeuioaueoiauoeiuaioeaoiueoiaueoiuaoeuoaiueoiaueoiaueioua

        Huh. On the physical DECTalk I played with, if you sent JUSt the vowels, it would pronounce them like:

        • a o i u e i o a u e o i u a o i e u a o i e u i o

        Actally, even that is different because it used to leave pauses between the letters, and now it speechifies them. lol.

        What else? Oh, this was always a winner:

        • sheeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeit

        Yeah, this version is a little different, but it is decently close. The voice is right and a lot of the cadence is, but it feels like maybe a later version than I had. lol

        And yes. We were more easily entertained in those days. XD