Digital resources in the Social Sciences and Humanities OpenEdition Our platforms OpenEdition Books OpenEdition Journals Hypotheses Calenda Libraries OpenEdition Freemium Follow us

Talking About Muslims in Middle French: The Potential of Word-to-Vector Models for Studying Semantic Relationships in Medieval Languages

An illumination of the Great Khan at a banqueting table being served by his men above text in bastarda script.

by Kimberly Lifton

Medieval vernaculars are notoriously tricky for digital humanists to work with because they lack standardized spelling. Especially when using out-of-the-box libraries and software, most Natural Language Processing (NLP) techniques simply do not work well for medieval languages. However, word-to-vector models have the capacity to handle noise like spelling variants when trained on a significant number of words. As part of my PhD project, which examines the representations of Muslims in textual sources during the rise of the Ottomans in the fifteenth century, I have created custom word-to-vector models using Middle French texts. These models capture the constellations of Muslim representations in Middle French texts at the word level. My methodology considers the exploratory potential of word-to-vector models for shaping research questions in a process that Gabor Mihaly Toth has aptly described as “semantic wanderings.”1

Introduction

Word-to-vector or word embeddings is a technique in Natural Language Processing (NLP) where words are mapped to real numbers or vectors, which enables words to be represented in a continuous vector space that is capable of capturing words’ meanings and relationships in a corpus. My process draws on the word-to-vector techniques for historical research developed by Suphan Kirmizialtin and David Joseph Wrisley.2 I have created two word-to-vector models using Middle French sources, one for more traditional narrative manuscript sources and another for inventories. My choice to create separate models for different types of sources allows me to compare how authors and cataloguers used the same terminology for similar or different ends depending on their objectives. The word “sarrasinois” for instance might appear in a drastically different context in an inventory versus a travel narrative.

For the model trained on narrative sources, I selected a total of 50 manuscripts from the fifteenth century containing different types of narrative texts that mention Muslims. For a full list, see the table below. My selection criteria were largely influenced by what is publicly available to download from archives. I attempted to limit the number of repeated texts to avoid over-representation. However, medieval texts are famously intertextual, which is something I want to capture in my results. Some of the manuscripts include other languages like Latin, Middle English, or Flemish. Although, since the presence of other languages was relatively minimal, they do not appear to have influenced the model. For the inventories model, I used a total of twenty fifteenth-century inventories from France and the Low Countries all in Middle French but containing Latin phrases. Ultimately, the two sets of training data are relatively small compared to fastText’s pre-trained models for modern languages but demonstrate some promising results.

Workflow

I began the process of creating a custom fastText model by using a layout analysis model and a premade Kraken Handwritten Text Recognition (HTR) model, TRIDIS v2, to extract machine-readable text from digitized manuscripts on the eScriptorium platform. TRIDIS v2 is an HTR model for multilingual Latin script from the eleventh to the sixteenth century trained on semi-diplomatic transcriptions (abbreviations were expanded).3 The training and validation dataset consisted of 1,855 pages with 120,000 lines of text and an additional 420,000 lines from a GAN model imitating medieval handwriting. The model has an accuracy rate of  96.8%, but this of course differs in practice from manuscript to manuscript. For instance, on fol. 6r of Paris, Bibliothèque de l’Arsenal, Ms. 5074 réserve, which I have selected at random from my automated transcriptions, the HTR transcription of the first few lines reads:

fait il Si uous diray en quelle maniere et que uous
et moy auons a faire uray est cõme uous scaues q̃
lempereur a guerre au duc regnault et a nous de
si longue main que zay paour que ia paix ne sen

A manually corrected transcription reads:

fait il Si uous diray en quelle maniere et que uous 
et moy auons afaire uray est cõme uous scaues q̃
lempereur a guerre au duc Regnault et a nous de 
si longue main que jay paour que ja paix ne sen

In the case of this particular Burgundian Gothic Cursive or bastarda hand, the HTR model demonstrates confusion with the letter j and is not case sensitive (r vs R). Please note that I do not consider differences between u and v to be errors, as is common practice in transcription conventions. While issues with identifying the letter j could impact my results, case sensitivity will not. The quality of a few HTR transcriptions was also influenced by image-quality, like the transcription for BnF Dupuy 255, which I could only access as a digitized microfilm scan. Here is an example of the HTR transcription from fol. 1r:

de rhodes est assise aulont de lisle en
contre leuant et tremontanc et nest poĩt
des plus grandes ne aussi des mendres.
Et est toute Ronde Reserut que le port de lamer in

A manually corrected transcription reads:

de Rhodes est assise aulont de lisle en 
contre leuant et tremontane et nest poĩt 
des plus grandes ne aussi des mendres 
Et est toute Ronde Reserue que le port de lamer en

While, in this case, the accuracy rate is not a significant inhibitor to my results, the worst transcription of the fifty manuscripts was for BnF Français 1489. The scanned microfilm images seem to have further degraded in quality upon importing them into the eScriptorium interface. Here is an example of the HTR transcription from a randomly selected section of fol. 1r:

Que li rois d'Engletee de say char engendra
du roy amani questoce gouverna
Eun pour lamour de Deu le lor mahonnli a
Et depuis sur paienr si bien il e porta

Compare this to a corrected transcription:

Que li Rois dengleterre de sar char engendra 
Et du Roy amauri quescoce gouverna 
Qui pour lamour de dieu le loy mahom laissa 
Et depuis sur paiens si bien il se porta

Even in the instance of BnF Français 1489, the HTR transcription is still usable. The flexibility of the word-to-vector model also helps with the noise created by minor transcription errors of one or two characters.

After using this HTR model in the eScriptorium platform to create machine-readable text files, I cleaned the text files of punctuation (aside from hyphens), capital letters, and line breaks for training data. The resulting model for narrative sources was trained on a total of 2,700,123 words and 12,142,943 characters (excluding punctuation and spaces). I used a Skipgram model for unsupervised training with 100 dimensions and 10 epochs. The number of dimensions is dependent on the amount of data available for the specific use case. 300 dimensions is a common paramterization for modern language pre-trained fastText models. However, there is far less data for Middle French making 100 dimensions more suitable in this instance (to avoid overfitting on too small a dataset).

For the model trained on inventories, I used OCR transcriptions from scanned editions of the texts, which I obtained through Gallica, Archive.org, and Hathi Trust. For this model, I also trained an unsupervised Skipgram model on a total number of 2,465,953 words (23,738 unique words) with 100 dimensions and 10 epochs. Although the inventory training data was substantially smaller than my narrative source training data, the quality of machine-readable text was higher.

To evaluate the accuracy of the model, I looked at the quality of the word embeddings that the model generated through intrinsic evaluation methods, including evaluating the model’s outputs for nearest neighbor results. The nearest neighbor results demonstrated the model’s ability to identify spelling variants accurately as well as synonyms and antonyms. For the narrative sources model, here is a list of the top 30 words with the highest cosine similarity to “tartar”:

Words associated with 'tartar':
0.9335034489631653: tartars
0.9177899360656738: tartarie
0.902688205242157: tarta
0.8939720392227173: tartarins
0.741866409778595: tars
0.6958993673324585: tartres
0.6886648535728455: tarse
0.6694226264953613: taitars
0.6531116366386414: tartre
0.6460373997688293: sartazins
0.6442888975143433: tarie
0.6340306401252747: baldach
0.61935955286026: kiaan
0.6164014339447021: sairazins
0.6123291254043579: kan
0.6096377372741699: soldan
0.603230357170105: syrie
0.6014472842216492: sauazins
0.598444938659668: caan
0.5979862809181213: empereour
0.597381591796875: kaan
0.5945258736610413: halappe
0.5937740802764893: mango
0.592740535736084: liaan
0.5844046473503113: seigneurie
0.5833797454833984: gypte
0.581447958946228: mangy
0.5791760683059692: grecs
0.5790480971336365: turquenians
0.5782543420791626: tart

The first nine words are spelling variants of Tartar or Tartary. In many of the instances where “tars” and “tarie” appear as individual words, they are separated from the first half of the word “tartars” and “tartarie” respectively. An example of this is the line “iii b uant le souidan de turquie ot assemble sonost de toutes pars et uint et se combati aux tar tars en unlieu qui est nonme cosadathgrant fut 3 la bacaille et asses cnyot de mors dime part et cautre mais.” The ability of the model to identify subwords makes it capable of coping with these types of transcription issues.

Subwords for 'tartar': (['tartar', '<ta', '<tar', '<tart', '<tarta', 'tar', 'tart', 'tarta', 'tartar', 'art', 'arta', 'artar', 'artar>', 'rta', 'rtar', 'rtar>', 'tar', 'tar>', 'ar>'], array([ 11178, 624971, 350665, 353353, 792128, 808467, 426671, 552850, 507458, 1840673, 1673096, 1556432, 158876, 160941, 1241347, 720045, 808467, 1528573, 1832099]))

However, not all spellings of “tars” refer to the Tartars. Some represent variants of the word “tard.” It is difficult to determine to what degree the model picks up on homographs, words that are spelled the same but have different meanings, because “tars” is not a common spelling of “tard” in the corpus.

Additionally, variations of the place name Tarsus may also be muddled with variations for the realm of Tartary because the two have similar or even the same spellings and appear in similar contexts. While not necessarily synonyms, “sartazins,” “sauazins,” and “sairazins,” all variants of Saracen, are close in contextual meaning to Tartar across the training data. Other words like “kiaan,” “kan,” “caan,” and “kaan” (khan) commonly appear in phrases like “grant can” or “mango can” to refer to the ruler of the Mongol khanate. Based on context, “liaan” is a mistranscription of “kaan” as well. While the amount of training data and quality of machine-readable transcriptions are limitations, overall, results suggest that the models are viable for exploratory purposes.

Results

After training a custom fastText model on Middle French texts, I ran a series of semantic queries for words relevant to my study and mapped their semantic relationships to other words. The proximity in a word vector can reveal synonyms, antonyms, word analogies, or common contexts. I visualized these relationships using Principal Component Analysis (PCA) graphs for reducing the high-dimensional word vector data generated by the model to two-dimensional space. Plotting word vectors in a two-dimensional space allows me to capture the constellations of rich semantic structures associated with demonyms for Muslims and territories ruled by Islamic sultanates.

A Principle Component Analysis graph showing vectors with cosine similarity scores of >=0.6 to “orient” and its spelling variants.
PCA graph of “orient” and its spelling variants using the cosine similarity of vectors from the inventories model

While some of my findings are expected—such as the tendency for Saracens to be discussed in relation to Christians whereas Christians are discussed in a broader range of contexts—others are more surprising. For instance, results from the inventories model demonstrate that terms related to orient and outremer tended to refer to natural resources extracted from the environment like gemstones, while Sarrasinois or variations of sarrazen tended to refer broadly to man-made goods from Islamic kingdoms. However, man-made goods more commonly co-occurred with the adjectival forms of demonyms, like Turkish and Moorish, or with locationally specific designators, like from Damascus or from Alexandria. These lexicons suggest that applying the terms orientalist, oriental, Eastern, or Orientalism to the Middle Ages is too totalizing to accurately describe how Latin Christendom perceived Muslim peoples and the geographies they resided in. Similar trends are mirrored in the narrative sources as well.

Taking inspiration from Max M. Louwerse and Nick Benesh’s “Representing Spatial Structure Through Maps and Language: Lord of the Rings Encodes the Spatial Structure of Middle English,” I have also used the results of the narrative sources model to create a “cognitive map”—a mental representation of spatial information about places.4 In the article Louwerse and Benesh demonstrate that spatial mental representations can be mapped from linguistic sources through statistical linguistic frequencies. Their results suggest how the physical distance between locations in Middle Earth can be “estimated by the lexical company they keep, leading to the conclusion that language encodes spatial structure.”5 This inspired me to look at the cosine similarity of location names in narrative sources with the question “How directly will my results correspond to a geographic map of the world, a Ptolemaic map, or a T-O map?” in mind.

A T-O map oriented with the east at the top and the west at the bottom showing the tri-partite organization of the world into Europe in the lower-left corner, Africa in the lower-right corner, and Asia in the top half.
T-O Map, New Haven, Beinecke Rare Book and Manuscript Library, Beinecke MS 358, fol. 74v

To display my results, I created two PCA graphs, one which shows vectors for different spelling variants of place names and another that calculates the centroid of the spelling variant vectors. The results show that England, Portugal, Spain, Bohemia, France, and specific city-states in Italy are clustered together in a “Europe” cluster. Hungary hovers further off to the side from this cluster. The PCA graph that contains spelling variant vectors shows that this is because some variants of Hungary hover closer to Syria, Mesopotamia, and Trebizond, possibly because of Hungary’s role in the late medieval crusades. On the outskirts of the European cluster is another smaller cluster of “in-between” places—Italy, Cyprus, Africa, and Israel (the traditional center point of the T-O map). Notably, Jerusalem and Judea are not in the center between the “Europe” cluster and the other clusters containing a mix of locations from what is usually labeled “Asia” and “Africa” on a T-O map. This may be the result of pilgrimage narratives that would have clustered location terms for places along the common pilgrimage route—Galilee, Bethlehem, Nazareth, Jericho, etc. The clusters on the bottom half of the PCA centroid graph containing Persia, Tartary, Armenia, Georgia, Turkistan, Türkiye, and Syria are more perplexing. These clusters likely represent a spatial thinking less influenced by the standard pilgrimage route from Venice to Jerusalem and more general.

A Principal Component Analysis graph of place names where centroids are calculated for spelling variants of place names.
PCA of Centroids for Each Geographical Term
A Principal Component Analysis graph that includes vectors for spelling variants of place names that are color-coded by the place the spelling variant represents.
PCA of Word-to-Vec Cognitive Map

Potentially the most interesting finding from this lower half of the centroid PCA graph is the cluster containing “Orient,” “Occident,” and “Indie” (a term that refers to the Tribus Indies, the geographical Orient beyond Muslim-occupied lands). One might expect Occident somewhere amid the “Europe” cluster but instead it sits right beside Orient, indicating that Occident is used as a complimentary term to Orient and rarely independently. As a whole, the vectors do not spatially superimpose onto either the Ptolemaic map, which became popular in Latin Christendom during the fifteenth century, or the T-O map. They represent a more complex web of spatial thinking comprised of several different discourses, some on pilgrimage others on crusade. While ideas about the spatial organization of “Europe” were relatively stable across these discourses, as my dissertation shows, spatial ideas of “Africa” and “Asia” were more fluid. They resist modern East-West binaries in their multiplicity.

The digital humanities methodology that I have presented in this blog is exploratory, and, while it exposes the complexities of spatial thinking in the fifteenth century, it does not broach the question “why?” For this, I turn to close readings of my sources in a process of oscillated reading that moves between the “distant” reading of digital humanities and traditional “close” reading.6 In zooming in and out on my sources, my dissertation pieces together why the fifteenth-century Middle French image of the world constantly shifted as the borders of the known world expanded.


Appendix

List of manuscripts used for narrative sources model training data:

Manuscript Lines Words Characters
BnF, Arsenal 5072 7556 67337 298324
BnF, Arsenal 5073 9952 83375 377390
BnF, Arsenal 5074 7443 62889 282219
BnF, Arsenal 5075 7251 59794 270599
Beinecke, MS 1346 349 2336 11130
BnF, Dupuy 255 6314 54117 246337
BnF, Français 810 1172 6021 27633
BnF, Français 92 14741 139622 637493
BnF, Français 358 11221 52056 239067
BnF, Français 764 20124 170726 703439
BnF, Français 852 13170 111720 504041
BnF, Français 947 12166 58919 272488
BnF, Français 1278 7111 69901 315623
BnF, Français 1380 7664 43483 191096
BnF, Français 1489 3898 38435 165996
BnF, Français 1506 14309 104188 475554
BnF, Français 1721 6042 39319 177393
BnF, Français 2200 3560 21643 102770
BnF, Français 2810 12471 136670 597185
BnF, Français 2868 8059 45240 203902
BnF, Français 5063 1622 20757 90911
BnF, Français 5593 950 4584 22204
BnF, Français 5594 21691 111592 503650
BnF, Français 5646 1057 7263 35610
BnF, Français 6440 25092 110541 509820
BnF, Français 6487 1837 25264 118482
BnF, Français 9087 6034 50648 223412
BnF, Français 9201 5963 58319 272152
BnF, Français 9737 8384 59454 288057
BnF, Français 11594 6851 40018 177228
BnF, Français 11610 5991 44272 201785
BnF, Français 12201 3126 24732 117056
BnF, Français 12572 3243 25721 115456
BnF, Français 12574 1710 12110 53664
BnF, Français 13235 7641 28444 133543
BnF, Français 15217 3179 28973 127020
BnF, Français 19170 12161 65206 296045
BnF, Français 20055 3893 23342 113525
BnF, Français 24210 10070 96762 405137
BnF, Français 24371 7049 56585 259434
BnF, Français 25295 950 4584 22204
BnF, Français 25434 4484 24918 114736
Gent, Universiteitsbibliotheek, MS 2749/11 573 2983 13145
Gent, Universiteitsbibliotheek, MS 418 769 6547 28025
Lille, Ms. Godefroy 50 14270 117514 524452
Metz, Bibliothèque-médiathèque, MS 1562 9544 99361 439803
BnF, NAF 10057 11550 97338 446316
BnF, NAF 20960 4089 45236 209187
Princeton, Garrett 168 740 3353 16694
Newberry Library, Case, MS 54.5 3849 35911 164511

Featured Image: Jean le Long d’Ypres, Livre de l’Etat du grand Khan, BnF Français 2810, fol. 136v.

Cite this article as: Kimberly Lifton. Talking About Muslims in Middle French: The Potential of Word-to-Vector Models for Studying Semantic Relationships in Medieval Languages. DH Lab (Blog). https://dhlab.hypotheses.org/?p=4713.

  1. Gabor Mihaly Toth, “Women in Early Modern Handwritten News: Random Walks and Semantic Wandering in the Medici Archive,” Journal of Digital History 3.2 (2024). https://journalofdigitalhistory.org/en/article/jnkqqTTKW8km []
  2. Suphan Kirmizialtin and David Joseph Wrisley, “Exploring Gulf Manumission Documents with Word Vectors,” Journal of Digital Islamicate Research 2 (2024), 1-29; Also see, Avery Blankenship, Sara Connell, and Quinn Dombrowski, “Understanding and Creating Word Embeddings,” Programing Historian (2004). https://doi.org/10.46430/phen0116; Anton Ehrmanntraut, Thora Hagen, Leonard Konle, and Fotis Jannidis, “Type-and Token-based Word Embeddings in the Digital Humanities,” Computational Humanities Research (2021), 16-38; word-to-vector has also been used to trace change over time. For instance, see Jaap Verheul, Hannu Salmi, Martin Riedl, Asko Nivala, Lorella Viola, Jama Keck, and Emily Bell, “Using Word Vector Models to Trace Computational Change Over Time and Space in Historical Newspapers, 1840-1914,” Digital Humanities Quarterly 16.2 (2024); Melvin Wevers and Marijn Koolwen, “Digital Begriffsgeschichte: Tracing Semantic Change Using Word Embeddings,” Historical Methods: Journal of Quantitative and Interdisciplinary History 53.4 (2020), 226-243. []
  3. Sergio Torres Aguilar, “TRIDIS v2 : HTR model for multilingual medieval and early modern documentary manuscripts (11th-16th),” Zenodo, September 30, 2024. https://doi.org/10.5281/zenodo.13862096. []
  4. Max M. Louwerse and Nick Benesh, “Representing Spatial Structure Through Maps and Language: Lord of the Rings Encodes the Spatial Structure of Middle Earth,” Cognitive Science: A Multidisciplinary Journal 36.8 (2012): 1556-1569, https://doi.org/10.1111/cogs.12000. []
  5. Ibid., 1564. []
  6. For an argument on the benefits of this method see Inge van de Ven, “Too Much to Read? Negotiating (Il)legibility between Close and Distant Reading,” in Legibility in the Age of Signs and Machines (Leiden: Brill, 2018), 180-196. []

OpenEdition schlägt Ihnen vor, diesen Beitrag wie folgt zu zitieren:
DH Lab (30. Mai 2025). Talking About Muslims in Middle French: The Potential of Word-to-Vector Models for Studying Semantic Relationships in Medieval Languages. DH Lab. Abgerufen am 17. Mai 2026 von https://doi.org/10.58079/1416d


Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert

This site uses Akismet to reduce spam. Learn how your comment data is processed.