Unicode XOR Magic? Steganography with Arabic Diacritics

It piqued my interest one day, out of curiosity, to read into what kind of cryptographic or steganographic techniques Arab-Muslim scholars had developed, and I stumbled upon several sources, one of them being a section in al-Qalqashandi’s magnum opus ‘Subh al-A’sha’ (p.s: there is plenty of English literature about him, check Google Books!).

One story had particularly intrigued me, and fortunately enough, there already is a translation and study of the whole section by C. E. Bosworth. It can be found in the Journal of Semitic Studies (Vol 8, Issue 1, Spring 1963) under the title “The Section on Codes and Their Decipherment in al-Qalqashandi’s ‘Subh al-A’sha.”

Al-Qalqashandi relates the following story:

A certain king ordered his secretary to write a letter from him to one of his courtiers, blandishing him and setting his mind at rest, in order that he might seize him whenever opportunity arose. It happened that the secretary and the addressee were friends; so the secretary wrote the message as he was commanded, not deviating one jot from the usual epistolary form, except that when he wrote at the end, “If God, He is exalted, wills” (inshaAllah), he placed a shadda over the ’nuun’ of ‘in’ (innshaAllah).

When the courtier read it, he realized that this was no meaningless act on the secretary’s part, and he set about cogitating over its secret interpretation and significance. It dawned on him that the secretary was alluding to God’s words “Indeed, the assembly are conspiring together to kill you”¹ so he was on his guard and took precautions. The king got to know about his wary attitude, and suspected the secretary of having added something to the message which would convey what the king was planning.

So he summoned him and asked him about it, and ordered him to write out the message exactly as he had written it, without deviating from the original by one jot. The secretary wrote it out without changing any part of his version, even putting in the shadda on the ’nuun’. When the king read it and noticed the shadda, he angrily taxed the secretary with it, saying," What did you mean by that?". He replied, ‘‘I was alluding to God’s words ‘Indeed, the assembly are conspiring together to kill you’".

The king was pleased by this, and forgave him because of his truthfulness with him (or perhaps, “because of his faithfulness to his friend”).

And I thought to myself, “Now that’s interesting! Using harakat (diacritics) to convey a hidden message!” And surely enough I realized there has been a growing body of modern research on steganography using Arabic script², but the field is still quite new.

I went to brew myself a coffee, and while painstakingly preparing it, I was daydreaming of a scenario where our dear secretary couldn’t have been caught—a harakat-based plausible deniability of sorts. Around that time, I had also been reading about plausible deniability with XOR ciphers, and I was thinking to myself: “Could we combine the power of XOR ciphers with the inconspicuous steganographic properties of Arabic harakat?”

Thereafter I began experimenting—and I had a few inelegant or failed attempts that I need not mention. The most interesting failed attempt, however, is the one which accidentally led to the correct one. One pivotal question I had overlooked is that XOR involving Arabic messages is not as trivial as I thought it would be. In my pursuit of figuring out how to XOR Arabic characters in the first place, I opted for a raw Unicode codepoint XOR instead of a byte-level XOR (which I had been trying initially³). “Eureka!” I thought to myself, “now the XOR should work! Arabic XOR ciphers are becoming a thing!…” or not. Not too fast.

But I came looking for copper and found gold.

The “Problem” with Arabic XOR

Arabic letters occupy the Unicode Basic Arabic block at U+0621–U+064A —just 42 codepoints. XOR-ing any two Arabic codepoints produces values in 0x00–0x7F , which is nowhere near the Arabic letter range. In my initial desperate attempt to make Arabic XOR Arabic give Arabic messages with various workarounds, I didn’t realize I had accidentally stumbled upon the solution instead of a problem. “What does that range contain? Can’t I just use it?”

Luckily enough, I realized that the resulting range from XOR-ing two Arabic codepoints produces the entire lowercase a–z ASCII range and a few useful symbols: backtick, braces, pipe, and tilde. Digits (0x30–0x39) and uppercase (0x41–0x5A) land in a dead zone — but that didn’t matter. We had a full set of English letters to work with!

So what’s actually going on under the hood?

Arabic letters are in the Basic Arabic block: a = 0x06xx and b = 0x06yy. Then:

a ⊕ b = 0x00(xx ⊕ yy)

The high bytes cancel: 0x06 ⊕ 0x06 = 0x00. So the result lives in 0x0000–0x00FF, which contains ASCII lowercase (0x61–0x7A) plus a handful of other printable characters.

Now look at the inverse in that same model:

b = a ⊕ c

If a = 0x06xx (Arabic) and c = 0x00zz (ASCII), then b = 0x06(xx ⊕ zz) — still 0x06xx, still in the Arabic block. The inverse seems trivial if you understand XOR ciphers, but it carries a subtle problem which I discuss later on in “The Offset Problem” section.

letters = [chr(c) for c in range(0x0621, 0x064B) if chr(c).isalpha()]
xor_values = set()
for a in letters:
    for b in letters:
        xor_values.add(ord(a) ^ ord(b))
vals = sorted(xor_values)
print(f"Possible XOR values: {len(vals)} unique")
print(f"Range: {min(vals)}-{max(vals)}")
print()
printable = [(v, chr(v)) for v in vals if 32 <= v < 127]
print(f"Printable ASCII ({len(printable)}):")
for v, ch in printable:
    print(f"  {v:>3} = '{ch}'")
print()
digits = [v for v in vals if 48 <= v <= 57]
print(f"ASCII digits (0x30-0x39): {digits if digits else 'NONE - dead zone!'}")
lower = [v for v in vals if 97 <= v <= 122]
print(f"ASCII lowercase (a-z): {len(lower)} of 26 available")
print(f"  {''.join(chr(v) for v in lower)}")

- The First Attempt

For context for those who don’t speak Arabic: Arabic script has a built-in marking system called diacritics (harakat or tashkeel). These are small marks placed above or below letters to indicate vowels — fatha (◌َ), damma (◌ُ), kasra (◌ِ), sukun (◌ْ), and others. They live in the Unicode range U+064B–U+0652, immediately after the letter block. In practice, every Arabic letter should carry a diacritic. But Arabs, for brevity, almost always omit them. Partially diacritized text is common and not remotely suspicious — in fact it’s far less suspicious than fully diacritized text (and most native speakers hate reading fully diacritized text anyway).

My initial idea was straightforward. I needed an Arabic text that is fully diacritized and long enough to embed a secret message in. I picked the Mu’allaqa of Imru’ al-Qays:

The poem’s letters I would call M1. The secret message is M3 (English lowercase). We compute a key: M2[i] = M1[i] ⊕ M3[i]. Then find those key letters scattered among the diacritized letters of the poem. Strip all diacritics and re-apply them only on the letters that spell out the key. The result is a partially diacritized poem instead of a fully diacritized one:

The key here is the letters marked with diacritics: ق، ز، ن، ج, which are meaningless on their own.

The solver extracts the diacritized letters (the key), XORs them against the poem’s letters, and the English secret should fall out…

It worked! But with an inelegant catch for my chosen secret. You had to slide the key across the poem and check which non-zero offset produces printable ASCII. Only one did, and the correct answer was self-identifying.

- The Offset Problem

The offset problem arose because M1[i] ⊕ M3[i] doesn’t always produce a valid Arabic letter. About half the time, it lands outside the Arabic letter range — overshooting into the diacritics zone (U+064B and above) or undershooting to U+0620.

What I didn’t notice earlier in my initial analysis is that while Arabic ⊕ Arabic falls perfectly into the a–z block, the inverse doesn’t always cooperate. ASCII lowercase spans 0x61–0x7A. XOR-ing those low bytes against Arabic low bytes (0x21–0x4A) can produce values from 0x00 to 0x5F. The Arabic letter range only covers 0x21–0x4A of that — so results in 0x4B–0x5F land on diacritics (they’re right next door in Unicode), and 0x20 lands on an unused codepoint. The high byte (0x06) survives nicely, but the low byte is the one which is usually out of bounds.

For short messages this was manageable — you could try different offsets, starting the alignment at letter 0, 1, 2… until every position produces a valid key letter: M2[i] = chr(ord(M1[offset + i]) ⊕ ord(M3[i])). For example, mooninkstorm decodes smoothly at offset zero — it coincidentally bypassed the problem entirely and didn’t need the solution described in the next section.

I explored several ideas to make the operation always land in range, for e.g. modular addition instead of XOR or restricting the alphabets. I didn’t like any of them. I settled on keeping XOR and finding an elegant way around the offset problem.

The Breakthrough

I had been fixated on making M1 ⊕ M3 work with a pre-existing carrier text. But what if, instead of hoping the text cooperates, I generated a carrier text where the first N letters are guaranteed to be XOR-compatible with the secret? The result would then be perfectly invertible — Arabic ⊕ ASCII would always fall into the Arabic letter range (0x0621–0x064A) and the offset is naturally zero. It would always result in a valid key M2 rather than going out of our intended boundary.

For each position in the secret, there’s a set of Arabic letters that produce a valid key when XOR-ed. It turns out to be typically 9 to 19 valid letters per position out of 42. That’s plenty of room to write a natural Arabic sentence!

I gave the constraint map to Google’s Gemini Flash 3.5 with extending thinking as a prompt:

Write a single Arabic sentence that is exactly 29 Arabic letters long.
Letters only — spaces, diacritics, and punctuation do NOT count toward
the letter count.

Each letter position must use one of the allowed letters below.
The sentence must be natural, grammatical Modern Standard Arabic.

CONSTRAINT MAP:
  Position  0: ذرزشصضطعفقكلمنهىي
  Position  1: ذرزسشضطفقكلمنه
  ...

RULES:
- The sentence must contain EXACTLY 29 Arabic letters (no more, no less).
- Each letter must come from the allowed set for its position.
- Spaces, commas, periods, diacritics (tashkeel) are free, they don't count.
- Write natural Arabic, it should read like a real sentence.
- Do NOT include any non-Arabic characters.
- It must be the beginning of a kid's story. Do not begin it with a conjunction like وفي..

Return ONLY the Arabic sentence, nothing else.

It works! It found a perfectly natural Arabic sentence (“One day, the king said he was sad from a worry from before, but hope does not die after that.”):

فِي يَوْمٍ كَانَ المَلِكُ يَقُولُ إِنَّهُ حَزِينٌ مِنْ هَمٍّ مِنْ قَبْلِ، وَلَكِنَّ الأَمَلَ لَا يَمُوتُ بَعْدَ ذَلِكَ

I verified it with a script

$ python validate_carrier.py "فِي يَوْمٍ كَانَ المَلِكُ يَقُولُ إِنَّهُ حَزِينٌ ..."
VALID — decoded: superdashsecretdashlinkdotcom

And not only one! I was surprised at how consistently it could produce valid sentences. The constraint map gives enough room that even a standard LLM, without constrained decoding, can write natural Arabic that satisfies all positions.

I also tried constrained decoding with local models (JAIS-13B and Qwen3-8B with a custom LogitsProcessor that masks out invalid tokens at each position). The results were mixed: sometimes meaningless boilerplate, sometimes incoherent passages, sometimes non-Arabic words creeping in. In the end, I realized that prompting a frontier model with the constraint map and a validation script worked best. The local model idea works but isn’t feasible.

- The Rare Letter Problem

Some Arabic letters rarely ever have diacritics:

• ا (alef), the most common letter in Arabic, yet it almost never carries a diacritic
• ى (alef maksura), same story
• آ (alef madda, U+0622), rare to begin with
• ؤ (hamza on waw, U+0624), same story

If the XOR produced any of these as a key letter, the subsequence search would never find them in the diacritized pool, no matter how long the carrier text.

The fix was simple: add these four letters to the constraint map’s exclusion list. If an M1 letter at some position would produce one of them as a key value, that M1 letter is invalid for that position. The constraint map again shrinks slightly, but there’s still plenty of room for natural Arabic writing.

- The Subsequence Encoding Problem

With the first N letters of the carrier guaranteed to produce valid key letters, the next step was embedding the key into the text. The mechanism is the following: find the key as a subsequence among the diacritized letters of a longer text, mark those positions, and strip everything else.

In order to achieve that, this requires a big pool of characters where the subsequence could appear — which requires lengthy Arabic text as padding. It was surprisingly not difficult to achieve. Thanks to the frontier capabilities of LLMs at generating slop, I didn’t need to invent any filler text. A generic tale about King Saleem who dreams of flying or a monkey named Ameen exploring a forest was boring enough that any adversary should scroll past it. I admit that while it could have been a particularly fun creative writing exercise, I’m not a writer — I’m a steganographer!

A subtle but important property for the encoder (in order to maintain plausible deniability) is that the diacritics in the stego output must be linguistically correct. This wasn’t difficult to achieve as the linguistic validity is inherited from the carrier. The encoder starts with a fully and correctly diacritized Arabic carrier text. When it re-applies diacritics at key positions, it only keeps diacritics that were already there in the original (it never invents new diacritic placements).

- The Accidental Camouflage Layer

I should have realized this before, but Arabic text often contains Unicode marks beyond the standard diacritics, dagger alif (U+0670) in words like هٰذا, and alef wasla (U+0671) in ٱلملك. These are legitimate Arabic orthographic marks, but they fall outside the diacritic range my scheme uses (U+064B–U+0652). I decided that they were welcome to stay in the carrier text if they appeared, as it unintentionally creates a camouflage layer, it’s an interesting decoy. A solver who naively grabs “everything that looks like a diacritic” gets garbage. The exact Unicode range that matters are only the 8 regular Arabic diacritics (U+064B through U+0652).

The Final Pipeline

The complete system works like this:

- Encoding:

1. Choose a secret message (lowercase English a–z, plus `` { | } ~`).
2. Generate a constraint map: for each secret character, compute which Arabic letters produce a valid key when XOR-ed. Exclude letters that never carry diacritics (ا, ى, آ, ؤ).
3. Write (or generate) an Arabic opening sentence whose first N letters satisfy the constraints.
4. Pad with a longer fully-diacritized Arabic text, anything.
5. Compute the key: M2[i] = M1[i] ⊕ M3[i].
6. Find M2 as a subsequence in the diacritized letter pool.
7. Strip all diacritics, re-apply only at the subsequence positions.
8. Output: a partially-diacritized Arabic text that looks natural. It is a self-contained Arabic text file.

- Decoding:

1. Extract every Arabic letter from the text in order → M1.
2. Extract only the diacritized letters (those in U+064B–U+0652), in order → M2 (the key).
3. For each i: M3[i] = chr(ord(M1[i]) ⊕ ord(M2[i])).
4. Read the secret.

- Discussion:

The elegance of this system comes from the intersection of three points in my opinion:

The diacritic system: Arabic has a native, culturally standard way to mark specific letters without changing the text’s meaning. Partially diacritized text is unremarkable.

The ⊕ range property: Arabic codepoints XOR together to clean (a-z) ASCII. This is a lucky consequence of where Unicode placed the Arabic block.

It’s hard to detect: The diacritic placements are linguistically valid, unremarkable to the naked eye, and don’t raise any immediate suspicion due to the non-trivial XOR decoding scheme.

With that said, the intended adversary is a human reader or a naive grep (someone scanning the text and seeing normal Arabic with some diacritics). A more sophisticated adversary could look at the statistical distribution of diacritized letters. In natural partially-diacritized text, diacritics tend to cluster on ambiguous words and unusual terms. Here it is driven by the match against the key, which may raise some eyebrows.

The Magic Trick: A Demonstration

“Every great magic trick consists of three parts or acts: The Pledge, The Turn, and The Prestige.” — Christopher Nolan

The Pledge:

In the first act, the steganographer generates an ordinary Arabic text. They might ask you to inspect the text to ensure it is real, unaltered, and completely normal. However, as you the readers already know, nothing is quite what it seems.

The Turn:

The second act is where the magic happens.

Take the ordinary Arabic text and make it do something extraordinary. At this stage, you are looking for the secret, but you won’t find it because you aren’t actually looking. You don’t really want to know how it’s done; you want to be fooled.

$ python diasteg.py encode input.txt  "superdashsecretdashlinkdotcom"
Secret:     superdashsecretdashlinkdotcom
Key:        رلفقضإثطتمققظحرإةرللحقخخثفةخي
Diacritics: 32
Output:     stego.txt
Verify:     superdashsecretdashlinkdotcom
Match:      ✓

The Prestige:

But you wouldn’t clap yet…

The third and final act is “The Prestige,” considered the hardest part of the illusion. Making something disappear is never enough; you have to bring it back:

$ python diasteg.py solve stego.txt
superdashsecretdashlinkdotcom

This is the phase that delivers the astonishing twist, tying up the trick — M1 ⊕ M2 elegantly decodes into M3, leaving the audience in complete awe.

The full implementation (and the examples) is available at the diasteg repo. It takes a diacritized Arabic carrier text and a secret message and produces a stego text automatically.