A Test Word Selection and Optimization Method for a New Swedish Test of Phonetic Perception in Noise

Witte E, Köbler S, Ekeroot J, Möller C

Örebro University & Audiological Research Centre (ARC), Örebro University Hospital, Sweden

Visual Aid

If the text is difficult to read, you can activate a visual theme with more contrast and larger text.

Activate visual aid
Download poster as PDF

Download the poster as a PDF.

Download (3.2 MB)

Hypothesis

Our hypothesis is that a detailed analysis of non-acoustic factors influencing lexical access of candidate test words will prove to decrease the need for acoustic optimization and thereby increase both the validity and reliability of our new Swedish test of phonetic perception in noise.

Step 1 - Calculating Word Perception Predictors using Corpus Linguistics – Developing the ARC-list

Raw Word Type Frequency

The number of times each word type (spelling) occurs in a set of Swedish internet blog corpora from 1998-2015.Total corpus size: 536 866 005 tokens. (“Blog mix” available at https://spraakbanken.gu.se/eng/resources)

Zipf-Scale Value

(A new word frequency measure, developed by Van Heuven, et al. (2014))

Image of Zipf-Scale Value formula

𝑍𝑖𝑝𝑓𝑤 = The Zipf-scale transformed word frequency of word 𝑤
𝐹𝑟𝑒𝑞𝑤 = Raw word type frequency of word w in the corpus
𝑇𝐶 = Total number of tokens in the corpus
𝑊𝑇𝐶 = Total number of word types in the corpus

Neighborhood Based Probability

(Neighbor: a word that differs by one insertion, deletion or replacement of a phoneme)

Image of Neighborhood Based Probability formula

𝑃𝑁𝐷𝑃𝑤 = Frequency-weighted phonetic neighborhood density probability of word 𝑤
𝑍𝑖𝑝𝑓𝑤 = Zipf-value of word 𝑤
𝑛 = number of phonetic neighbors (index distance=1) of word 𝑤
𝑍𝑖𝑝𝑓𝑖 = Zipf-value the 𝑖th phonetic neighbor of word 𝑤

Phonotactic probability - Stress and syllable structure based

(The probability of the order of phonetic segments in a word, given the language as a whole)

Image of Phonotactic probability formula

𝑃𝑃 = Average normalized phonotactic probability of a word
𝑛 = number of phonemes in the word, including word start and word end markers
𝑝ℎ𝑖 = the 𝑖th phoneme in the word (including word end/start markers)
𝐼𝑆𝐿𝑝ℎ𝑖 = Intra-syllabic location of 𝑝ℎ𝑖 (Onset, Nucleus, or Coda)
𝑆𝑇𝑝ℎ𝑖 = Syllable type/position where 𝑝ℎ𝑖 occur (Stressed syllable, Ante-Stress, Inter-Stress, or Post-Stress syllable position)
𝑃*(𝑝ℎ𝑖 ⋂ 𝐼𝑆𝐿𝑝ℎ𝑖 ⋂ 𝑆𝑇𝑝ℎ𝑖) = the probability of the most probable phoneme given 𝑝ℎ𝑖 ⋂ 𝐼𝑆𝐿𝑝ℎ𝑖 ⋂ 𝑆𝑇𝑝ℎ𝑖

Spelling regularity

(The degree of spelling-to-pronunciation agreement)

Image of Spelling regularity formula

𝑆𝑅 = Average normalized spelling regularity of a word
𝑛 = number of grapheme-phoneme correspondences
𝑝ℎ𝑖 = the 𝑖th phoneme (/phoneme combination)
𝑔𝑖 = the 𝑖th grapheme
𝑙𝑖 = the first letter of the 𝑖th grapheme
𝑃𝑙𝑖 = the probability of the most probable phoneme given the grapheme initial letter 𝑙_𝑖

Grapheme-Phoneme Correspondences

The grapheme-phoneme correspondences are created using a custom made finite state transducer (FST), which uses a set of p2g rules in order to segment the spelling into an array of graphemes, corresponding to the phoneme array. The FST is capable of parsing silent graphemes (denoted ∅), testing for phonetic deletion processes, as well as skipping unparsable phonemes (denoted !).

Example words from the ARC list

Total word count = 816 404 words (9 433 monosyllabic)

Spelling Pronunciation Phonotactic Type Raw Word Type Frequency Zipf-Scale Value Neighborhood Density Probability Phonotactic Probability Spelling Regularity Grapheme-Phoneme Correspondences
baud b ˈ oː d CVC 5 1,05 0,02 0,86 0,85 b-b|au-o|d-d
deuce d ʝ ˈ uː s CCVC 74 2,14 0,15 0,83 0,68 d-d|!e-!ʝ|u-u|c-s|e-∅
dyks d ˈ yː k s CVCC 12 1,38 0,12 0,87 0,99 d-d|y-y|k-k|s-s
f ˈ ɛ̝ fː VC 13834 4,41 0,07 0,85 0,5 !-!ɛ̝|f-f
fjeld f ʝ ˈ ɛ̝ lː CCVC 8 1,22 0,23 0,89 0,79 f-f|j-ʝ|e-ɛ̝|l-l|!d-!
gicks ʝ ˈ ɪ kː s CVCC 62 2,07 0,03 0,93 0,97 g-ʝ|i-ɪ|ck-k|s-s
grapes ɡ r ˈ ɛ̝ ʝː p s CCVCCC 90 2,23 0,25 0,87 0,9 g-ɡ|r-r|a-ɛ̝ ʝ|p-p|e-∅|s-s
hyl h ˈ yː l CVC 8 1,22 0,02 0,86 0,99 h-h|y-y|l-l
jämnts ʝ ˈ ɛ̝ mː n t s CVCCCC 0 0,27 0,03 0,85 1 j-ʝ|ä-ɛ̝|m-m|n-n|t-t|s-s
klan k l ˈ ɑː n CCVC 333 2,79 0,06 0,95 0,99 k-k|l-l|a-ɑ|n-n
kåhl k ˈ oː l CVC 0 0,27 0,05 0,88 0,95 k-k|å-o|h-∅|l-l
muf m ˈ ɵ fː CVC 181 2,53 0,06 0,83 1 m-m|u-ɵ|f-f
plaggs p l ˈ a ɡː s CCVCC 5 1,05 0,06 0,85 0,97 p-p|l-l|a-a|gg-ɡ|s-s
plump p l ˈ ɵ mː p CCVCC 416 2,89 0,14 0,87 1 p-p|l-l|u-ɵ|m-m|p-p
pysch p ˈ ʏ ʂː CVC 32 1,79 0,11 0,78 0,89 p-p|y-ʏ|sch-ʂ
schack ɧ ˈ a kː CVC 1245 3,36 0,04 0,88 0,86 sch-ɧ|a-a|ck-k
sen s ˈ ɛ̝ nː CVC 893115 6,22 0,05 1 0,98 s-s|e-ɛ̝|n-n
sprack s p r ˈ a kː CCCVC 3034 3,75 0,22 0,94 0,99 s-s|p-p|r-r|a-a|ck-k
törsts t ˈ œ ʂː ʈ ʂ CVCCC 0 0,27 0,08 0,89 0,86 t-t|ö-œ|rs-ʂ|t-ʈ|s-ʂ

What’s more in the ARC-list?

  • Vitevitch & Luce’s (1999), as well as Storkel’s (2004) types of phonotactic probability measures.
  • Spelling regularity calculated according to Berndt, Reggia, & Mitchum (1987).
  • PLD1 neighbors
  • Number of senses from SALDO (Borin, Forsberg, & Lönngren 2013)
  • Contextual diversity
  • Word classes and Lemmas, pitch accent, letter count, grapheme count, di-graph count, tri-graph count, proportion of upper case initial letter, special characters, phoneme count, syllable count, index of primary and secondary stressed syllables...
  • Marking of abbreviations, acronyms and foreign words

The present project aims to develop a highly reliable, and clinically suitable, Swedish language test of phonetic perception in background noise for adult persons with hearing loss. The test will be a multiple choice rhyme test based on test word groups consisting of real words with minimal phonemic contrast.

In order to maximize test reliability, two types of test stimuli optimization procedures will by utilized. Both of these aim to equalize the difficulty level of all contrasted test items:

  1. Word perceptual probability optimization, in which the minimally contrasting words that have the least variation in word frequency, phonetic neighborhood density, phonotactic probability and spelling regularity, in the whole of the Swedish language are identified.
  2. Acoustic optimization, in which the sound levels of each contrasting phoneme (rather than of the whole test word, which is common procedure) are experimentally adjusted until a predefined degree of intelligibility is reached.

Test word candidates are selected from a new Swedish word list, the ARC-list (Audiological Research Centre, Örebro), containing several types of measures which have previously proven to affect word recognition speed and accuracy in other languages, as well as some newly developed ones.

We hypothesize that the careful test word selection method described here will prove to decrease the amount of acoustic optimization needed to achieve equal perceptibility within the test word groups, and thereby increase both the reliability and the ecological validity of the test material.

Step 2 - Selecting The Most Optimal Test Word Group

Image of the planned layout of the computerized test
Planned layout of the computerized test of phonetic perception in noise.
  1. Computerized identification of groups with minimal phonemic contrast.
  2. Manual selection of groups containing the desired phonetic contrasts.
  3. Selection of groups with the least intra-group variation in the word perception predictors:
    • Zipf-scale transformed word frequency value (r2 = 51%)
    • Frequency-weighted phonetic neighborhood density probability (r2 = 54%)
    • Average normalized phonotactic probability (r2 = 33%)
    • Spelling regularity (r2 = 34%)

(Within brackets: the contribution of each predictor type to the average intra-group variation, as determined by Pearson’s correlation.)

Image of number of groups of monosyllabic words with minimal phonetic variation

Method of calculating average estimated intra-group variation in word perception probability:

  1. Coefficient of variation (CV = 𝜎 / 𝜇) is calculated for each group of words with minimal phonemic variation.
  2. Each prediction variable is rank transformed between all groups, to enable approximately equal influence between predictor variables.
  3. Average intra-group variation in word perception predictors.

Example groups

Green = Low variation. Suitable as test word groups.

Red = Large variation. Not suitable!

Average intra-group variation Rank order transformed coefficients of variation Homophone Count Homograph Count Minimally contrasting phonemes Candidate Test Words
Zipf scale Neighbor-hood Phonotactic probability Spelling regularity
1024 1000 764 2114 217 3 0 b f k p s blint [blˈɪnːt], flint [flˈɪnːt], klint [klˈɪnːt], plint [plˈɪnːt], slint [slˈɪnːt]
1685 1875 2472 1756 638 0 1 ∅ f h m v int [∅ˈɪnːt], fint [fˈɪnːt], hint [hˈɪnːt], mint [mˈɪnːt], vint [vˈɪnːt]
1923 2475 2196 704 2317 1 2 a ɛ̝ ɪ ɵ ʏ lasts [lˈasːts], lästs [lˈɛ̝sːts], lists [lˈɪsːts], lusts [lˈɵsːts], lysts [lˈʏsːts]
2036 2719 2737 584 2104 0 0 b d f l t bukt [bˈɵkːt], dukt [dˈɵkːt], fukt [fˈɵkːt], lukt [lˈɵkːt], tukt [tˈɵkːt]
2573 2972 1586 3901 1833 2 0 ∅ l r s t stå [stˈoː∅], stål [stˈoːl], står [stˈoːr], stås [stˈoːs], ståt [stˈoːt]
2813 2020 3089 4363 1779 0 2 d k n p v lids [lˈiːds], liks [lˈiːks], lins [lˈiːns], lips [lˈiːps], livs [lˈiːvs]
3015 2161 1124 5504 3270 2 0 ɑː eː uː ʉː øː dag [dˈɑːɡ], deg [dˈeːɡ], dog [dˈuːɡ], dug [dˈʉːɡ], dög [dˈøːɡ]
3386 4331 4826 2354 2031 0 0 d k l m p dusts [dˈɵsːts], kusts [kˈɵsːts], lusts [lˈɵsːts], musts [mˈɵsːts], pusts [pˈɵsːts]
3860 5120 4753 1824 3743 2 0 a ɔ ɛ̝ ɪ ɵ tapps [tˈapːs], tops [tˈɔpːs], täpps [tˈɛ̝pːs], tips [tˈɪpːs], tupps [tˈɵpːs]
4144 2887 2914 5681 5092 2 0 ∅ r ʂ t v u [∅ˈʉː∅], ur [∅ˈʉːr], urs [∅ˈʉːʂ], ut [∅ˈʉːt], uv [∅ˈʉːv]
4477 4848 5400 3231 4428 0 1 ɕ f ɧ s t kyls [ɕˈʏlːs], fylls [fˈʏlːs], skylls [ɧˈʏlːs], sylls [sˈʏlːs], tylls [tˈʏlːs]
5011 5651 5152 4012 5228 0 0 fː ʝː kː nː pː tafts [tˈafːts], tights [tˈaʝːts], takts [tˈakːts], tants [tˈanːts], tappts [tˈapːts]
5558 5778 5779 5325 5349 0 1 d m n ʂ v dåds [dˈoːds], doms [dˈoːms], dåns [dˈoːns], doges [dˈoːʂs], dovs [dˈoːvs]

References

  • Berndt, R. S., Reggia, J. A., & Mitchum, C. C. (1987). Empirically derived probabilities for grapheme-to-phoneme correspondences in english. Behavior Research Methods, Instruments, & Computers, 19(1), 1-9. doi: 10.3758/bf03207663
  • Borin, L., Forsberg, M., & Lönngren, L. (2013). SALDO: a touch of yin to WordNet’s yang. Language Resources and Evaluation, 47(4), 1191-1211. doi: 10.1007/s10579-013-9233-4
  • Storkel, H. (2004). Methods for minimizing the confounding effects of word length in the analysis phonotactic probability and neighborhood density. Journal of Speech Language and Hearing Research, 47(6), 1454-1468. doi: 10.1044/1092-4388(2004/108)
  • van Heuven, W. J., Mandera, P., Keuleers, E., & Brysbaert, M. (2014). SUBTLEX-UK: a new and improved word frequency database for British English. Quarterly Journal of Experimental Psychology (2006), 67(6), 1176-1190. doi: 10.1080/17470218.2013.850521
  • Vitevitch, M. S., & Luce, P. A. (1999). Probabilistic Phonotactics and Neighborhood Activation in Spoken Word Recognition. Journal of Memory and Language, 40(3), 374-408. doi: 10.1006/jmla.1998.2618

Some significant source material (e.g. spellings and phonetic transcriptions) for the ARC-list are derived from: NST Lexical database for Swedish. Clarino NB – Språkbanken. Oslo: Nasjonalbiblioteket, available under Creative Commons ZERO license (CC-ZERO) at: http://www.nb.no/sprakbanken/show?serial=oai%3Anb.no%3Asbr-22&lang=en

Contact information

E-mail address for correspondence, erik.witte@oru.se

Relaterad personal: Erik Witte, Claes Möller, Susanne Köbler, Jonas Ekeroot