2.1. Stress in Brazilian Portuguese

Phonetically, duration has been widely described as the most robust acoustic correlate of primary word-level stress in BP (e.g., Barbosa, Eriksson & Åkesson 2013; Major 1985; Massini 1991), while F0 and intensity are less reliable correlates. In Arantes, Lima & Barbosa (2012: 17) the authors mention that vowels in stressed syllables have higher mean spectral emphasis than those in other positions of the word, which could thus indicate that it is a correlate of primary word stress. In later work, Barbosa, Eriksson & Åkesson (2013) examined duration, F0 standard deviation and spectral emphasis values for three different speaking styles in BP and found out that duration reliably distinguishes stressed vowels from unstressed ones. The authors also found that posttonic vowels have significantly lower mean values of spectral emphasis when compared to the ones in pretonic and stressed syllables (also refer to Endnote 7). In addition, the authors report no difference in the mean duration of pretonic and posttonic vowels in their study. This latest finding could indicate that secondary stress might not be phonetically expressed through differences in vowel duration. With respect to spectral features, when looking at the vowel space of [i e a o u] in polysyllabic words that bear penultimate stress, Arantes (2011) found that such space is maximal for stressed vowels, gradually contracting in pretonic positions.

As mentioned in the introductory section above, predictions about the locus of secondary stress (SS) fall outside of the scope of this inquiry. Nevertheless, within the realm of binary classification, secondary stress can potentially become a confounding factor, for example, if one contemplates the possibility that secondarily-stressed vowels, if or when phonetically realized, may be (acoustically) more similar to primarily-stressed vowels than to unstressed vowels. In this sense, secondary stress becomes relevant herein, warranting a very brief review of the literature.¹² There are competing narratives about both the locus and the phonetic nature of SS in the language: for Major (1985), all pretonic syllables of a word in BP bear secondary stress and all posttonic syllables are unstressed. Collischonn (1994), on the other hand, proposed that secondary stress in BP is binary in nature and that the domain for its assignment is the portion of the word located to the left of the stressed syllable. Barbosa et al. (2004) presented evidence that suggests that secondary stress is equivalent to word-initial prominence only. De Moraes (2003) found out that secondary stress was perceived regularly in words with more than one pretonic syllable and that the locus of perceived SS admitted variation, but the author found no stable correlate for it (de Moraes 2003: 2066). In Arantes & Barbosa (2008; 2006), the authors propose that secondary stress is best described as phrase-initial prominence and investigate different correlates: duration and pitch accent excursion (2006), and F1 and spectral correlates (2008). Abaurre & Fernandes-Svartman (2008), analyzed text read by 17 speakers and argued for the binary nature of SS in BP, noting that processes of vowel sandhi within the prosodic word, at the lexical boundary, tend to optimize such binary organization. This brief summary illustrates that there is no single answer to the question of where in a word SS falls in Brazilian Portuguese, nor to what its acoustic correlates are in the language. Since, as mentioned above, SS could become a potential confounder for the classifier, the matter needs to be addressed by methodological considerations, which will be discussed in the Methodology section below.

Phonologically,¹³ Portuguese is a bounded stress language, where stress is contrastive and predictable in so much as it is (mostly) restricted to the three right-most syllables of the word. Two possible violations to this constraint have been described: Pereira (2007), points out that the three-window constraint is violated when pronouns cliticize to verbs (example (3) below),¹⁴ and Lee (2007), notes that the epenthesis rule of BP, which inserts the front high vowel [i] to break-up disallowed consonant clusters as a means of recovering from phonotactic violations,¹⁵ may also generate words that display pre-antepenultimate stress (example (4) below). The near-minimal triplet of nouns (1) and the minimal triplet of words belonging to distinct lexical categories (2) shown below exemplify contrastiveness in the language.

Besides indicating word contrast within and between lexical categories as exemplified above, primary stress also expresses inter-paradigmatic contrast in BP, marking grammatical tense in verbs, as illustrated by the 3^rd person plural of the past perfective (penultimate stress) and of the future (ultimate stress), shown in (5).

Three generalizations about primary stress placement in Portuguese are broadly accepted: (i) antepenultimate is an exceptional pattern, no longer productive in the language; (ii) there is a preference for penultimate stress, unless (iii) the last syllable of the word is heavy—whereby a syllable is heavy if it is closed or if it possesses a bimoraic nucleus, which in the language can be an oral or a nasal diphthong or a nasal vowel (Wetzels 2007).

These generalizations however, do not make the task of describing the mechanism(s) that govern primary stress assignment in the language any more amenable, as made plain by the ample literature debating the topic, which dwells around a number of issues: while stress falls on the penultimate syllable in about 62.5% of 150,000 non-lemmatized words in a (dictionary) corpus study of BP (Araújo et al. 2007: 42), as per generalization (ii), antepenultimate stress words still make up 12.2% of the total number of words in the dictionary and ultimate stress words add to 24.9% (Araújo et al. 2007: 42), so the language is neither straightforward trochaic nor straightforward iambic. With respect to generalization (iii), words such as sabiá and paletó shown in (1) and in (2) above demonstrate that stress may fall on the ultimate syllable even when it is light. Moreover, there are words in which the last syllable is heavy, but where stress nonetheless falls either on the penultimate or on the antepenultimate syllable, as well as words where the penultimate syllable is heavy where stress falls on the antepenultimate syllable, as illustrated in (6). These examples demonstrate that there are a number of exceptions to generalization (iii).

Generalization (i), or the idea that antepenultimate stress is an exceptional pattern, has been more recently brought into question by corpus work such as Araújo et al. (2008), Araújo et al. (2007) and Viaro & Guimarães Filho (2007), all of which provide evidence that shows not only that the number of antepenultimately stressed words is not exactly marginal in the language (about 12%, as previously mentioned), but also that these words have been entering the language as regularly as the other two stress patterns, from the IX to the XX centuries. The same studies also dispute the notion that antepenultimate stress is a by-product of prescriptive grammar, as opposed to an integral part of speakers’ knowledge, pointing out the lack of psycholinguistic evidence to support those claims.

Besides the exceptions found to generalizations (i)–(iii) just discussed, the complexity of the stress system in Portuguese in general, and in BP specifically, displays an additional facet in which, for most accounts, stress assignment in verbs appears to be subject to a set of rules that differs from the one acting upon non-verbs, and which is possibly conditioned by the language’s inflectional paradigms, as seen in (5) above and also in (7) below.

Given all of the facts just listed, it follows then that the task of defining the set of rules that govern primary word-level stress assignment in Portuguese presents a number of challenges. Straight arguments supporting either a syllabic or a moraic trochaic language, even though enticing given the stress system of Latin, are not tenable without added machinery. Sensitivity to quantity (QS), even if positionally restricted, would need to account for the irregular patterns of stress illustrated herein (and for the ones not included in this brief review). Given the nature of the data, while a sizable part of the literature subscribes to the view that stress is mostly predictable in Portuguese, authors like Câmara Júnior (1970) and Morais-Barbosa (1994) proposed that stress is unpredictable in the language, thus specified in the lexicon. With respect to sensitivity to lexical category, authors like Andrade & Laks (1991), Bisol (1992) and Lee (2007), among others, argue that stress is category-blind, while works like Garcia (2017), Hermans & Wetzels (2012), Lee (1997), Pereira (2007), and Wetzels (2007), among others, provide accounts where different rules govern stress in verbs and in non-verbs. Syllable weight is said to play a role for stress placement in non-verbs (Bisol 1992; Garcia 2017; 2019; Hermans & Wetzels 2012; Massini-Cagliari 1995; Wetzels 2007, among others), but it is generally accepted to be non-bearing for the verbal paradigm, where stress is said to be morphologically conditioned.

Some of the additional machinery proposed in previous literature to account for irregular patterns and exceptions herein discussed include: (a) the domain of application of stress rules, which has been proposed to be the word (e.g., Bisol 1992) or the derivational root (e.g., Andrade & Laks 1991), or even a different one for verbs and for non-verbs (e.g., Lee 1997; Pereira 2007); (b) the existence of catalectic consonants (e.g., Bisol 1992) which would explain stress placement in words like café [ka.ˈfɛ]; and (c) segmental and syllabic extrametricality, which would explain other irregularities such as antepenultimate stress (the last syllable is extrametrical), irregular patterns of penultimate stress, and stress placement in verbs (e.g., Bisol 1992).

Among recent accounts, Garcia (2017) used the Portuguese Stress Lexicon (PSL, see Garcia 2014)—which was built using the Houaiss dictionary (Houaiss at al. 2001) as its base—to perform a statistical study of stress placement in non-verbs. The author argues that Portuguese has a (QS) stress system, that sensitivity to weight is not restricted positionally in the word and, importantly, that it is gradient, not categorical, as previous analyses proposed. In the study, a negative correlation between weight and antepenultimate stress was found in the lexicon analyzed. Based on these findings, Garcia proposes a grammar where “stress is assigned based on a probabilistic distribution derived from the patterns present in the lexicon” (Garcia, 2017: 75).¹⁹ Most recently, Garcia (2019) used the same PSL to simulate lexica that better approximate the lexicon of an adult native speaker, thus creating a lexical baseline to be compared to forced judgements of speakers on a number of conditions related to weight effects in Portuguese. The author finds that both lexical statistics and the grammar have a role in the phonological learning, and that in the case of antepenultimate syllables, lexical statistics and the grammar are at odds, since in the former it was found that there is a negative correlation between heavy syllables in antepenultimate position and antepenultimate stress, but the author’s experimental studies showed that speakers did not generalize this negative correlation to the nonce words tested. Naturalness would then be the mediator between lexical statistics and the grammar in the case of antepenultimate stress (Garcia, 2019: 636).

It is worth noting that, in addition to primary and secondary stress, Brazilian Portuguese has been reported to display an asymmetry between pretonic and posttonic syllables (e.g., Major 1985; Câmara Júnior 1970), whereby vowels in posttonic position are always unstressed. This piece of information is important because it plays a key role in determining how the results of the classification experiments are to be computed, a matter discussed in more detail in the Methodology section.

2.2. The West Point Brazilian Portuguese Speech Corpus

The West Point Brazilian Portuguese Speech corpus (LDC2008S04–Morgan et al. 2008), hereafter referred to as WPC, contains digital recordings designed and collected by the Department of Foreign Languages (DFL) at the United States Military Academy at West Point, and at the Center for Technology Enhanced Language Learning (CTELL). Recordings were made at the Brazilian military academy in Brasília in 1999.

The corpus is relatively balanced for gender, with sixty (60) female and sixty-eight (68) male native monolingual and bilingual speakers, who were recorded while reading a script containing two hundred isolated sentences. The WPC contains 131 declarative sentences, 49 interrogatives, and 20 negated sentences. The longest sentence in the prompts has ten words, and the shortest, one. In addition to containing a number of speakers sufficient to perform an ASR study, the corpus was chosen under the working assumption that the number of repetitions of the same word in the exact same prosodic context would be advantageous in training acoustic models of the language’s segmental material. Moreover, a corpus of read speech is likely as effective as a corpus of spontaneous speech when looking at the acoustic correlates of stress in BP. For example, the results found in Barbosa, Eriksson & Åkesson (2013: 285) pointed to “…a similar effectiveness of phrase reading and spontaneous styles in uncovering the word stress acoustic correlates in BP, at least for duration and F0 standard deviation.”

Out of the 128 speakers in the corpus, data from bilingual speakers—female speakers f40–f45, and male speakers m38–m52—were excluded from the study. Not all 200 sentences in the prompts were uttered by the 99 speakers included in the study, resulting in 7846 utterances, for a total of 39,894 word tokens.

There are 516 distinct word shapes in the WPC’s prompts,²⁰ a figure which is not thought to be a significant limitation to the study given the total number of word tokens in the corpus, adding to roughly 150,000 vowel tokens available to build the acoustic model. A brief summary of the corpus structure is given in Table 1.

The distribution of words in the corpus as a function of the number of syllables and of citation stress position is detailed in Table 2. Note that the number of syllables reported in this Table refers to a base pronunciation and not necessarily to the surface pronunciation produced by each speaker,²¹ which may be subject to epenthesis, syncope, and other phonological processes. Penultimate stress disyllabic words are the most frequent in the corpus, followed by ultimate stress disyllables, and by penultimate stress trisyllables. Note that the dashes (–) mean that the combination is not a logical possibility (as in antepenultimate monosyllable, for example), while the zeros (0) mean that there are no occurrences of that particular pattern in the West Point Corpus. In the Table, NA means that, although the pattern is a logical possibility, found in the language, and in the West Point corpus, it is not being included in the count for that particular cell in the Table. This is the case, for example, for disyllabic and trisyllabic function words.²²

2.3. Kaldi and ASR

Kaldi (Povey et al. 2011) is an open source Automatic Speech Recognition (ASR) toolkit, written in C++ and licensed under the Apache License 2.0. To the end user, from an input/output perspective, Kaldi is a collection of commands that can be used to both force-align speech and convert it into text. It is language-independent, and it encompasses normalization and transform algorithms to conduct speaker-independent and speaker-dependent research. The Kaldi toolkit can currently²³ extract standard MFCC (Mel Frequency Cepstral Coefficients) and PLP (Perceptual Linear Predictive) features from the speech signal. It models the acoustic data using either GMM-HMM (Gaussian Mixture Models-Hidden Markov Models) or SGMM/HMM (Subspace Gaussian Mixture Models-Hidden Markov Models). More recently, three DNN-HMM (Deep Neural Network-Hidden Markov Models) algorithms were added to the toolkit. Kaldi accepts, in principle, any Language Model (LM) that can be compiled into a Weighted Finite State Transducer (WFST).

To convert speech to text, in ASR²⁴ systems in general and in Kaldi specifically, the speech signal undergoes first feature extraction, a process wherein speech is windowed, typically in 25 millisecond slices, with one such slice being computed every 10 milliseconds. These discrete windows are then converted into high dimension (39 dimensions in Kaldi’s MFCCs), fixed size acoustic vectors, a process that can be completed using a few different encoding methods, among them, the two with which Kaldi works, MFCCs and PLPs. Each of the vectors generated in this fashion is known as an observation. Given a sequence O of such observations, the job of a speech recognition system is to find the maximally probable sequence of words W in language L that generated O, in other words, the ASR system needs to find the maximum likelihood of W given O.

$\widehat{W}=\mathrm{argma}{x}_{W}P(W|O)$ 1

Estimating P(W|O) directly—a pattern recognition problem—has not proven successful in the past,²⁵ so the naïve Bayes’ Rule, implemented employing generative models such as HMMs, is used instead.

$P\left(W\right|O)=\frac{P\left(O|W\right)P\left(W\right)}{P\left(O\right)}$ 2

The probability of an observation, P(O) is the same for each candidate sentence W, so the maximal value can be given by:

$\widehat{W}=\mathrm{argma}{x}_{W}P(O|W)P\left(W\right)$ 3

The likelihood of observing a given sequence of words P(W), where W ∈ L, is a prior calculated based on the Language Model (LM). The probability of a sequence of observations O given a sequence of words W, or (P(O|W), is calculated from the acoustic models (AM) and from the lexicon.

$P\left(O|W\right)={\displaystyle \sum }_{Q}P\left(O|Q\right)P\left(Q|W\right)P\left(W\right)$ 4

where Q = [q₁, q₂, …, q_T] is a sequence of acoustic states, one per frame. Resorting to an exceedingly abbreviated description here, P(O|Q) is calculated using information from the Mixture of Gaussian probability density functions (PDFs), and P(Q|W) is calculated using information from the Hidden Markov Models (HMMs).

Implementation-wise, in Kaldi (and also in other ASR systems), finding an answer to equation (2) is done by creating a weighted finite-state transducer (WFST) decoding graph H ° C ° L ° G, a composition of the HMM (H), the phone Context-dependency Model (C, see following paragraphs), the Language Model or Grammar (G), and the Lexicon (L). The most probable path through the decoding graph will give the most probable sequence of words that generated a given sequence of observations.

Within the architecture just discussed, P(O|W), or the likelihood that a sequence of observations be made given a sequence of words W in language L, is of immediate interest to the present study. In other words, we are interested in the decisions made during the forced alignment process, which is described hereafter in further detail. Figure 1 summarizes the discussion so far.²⁶

Figure 1

General architecture of an ASR system.

One crucial step of ASR is to extract meaningful information from the speech signal (the feature extraction process). During this process, the idea is to extract spectral information that maximizes phone recognition, so characteristics of the source (such as F0 and other details about the glottal pulses), which are not fundamental for phone detection are not directly modelled.

The most popular type of feature used in ASR is the MFCC (Mel Frequency Cepstral Coefficient), among other factors because it approximates human perception of speech better than other feature systems. This is partly because the windowed speech signal is filtered through a Mel-scaled filter bank, which introduces information about the human auditory perception into the model.²⁷ This process hence steers the model to focus on information that humans would find relevant in the speech signal.

MFCCs are described in some detail here as it is relevant for the present work to understand what information of the speech signal is kept in this compressed representation of it. Figure 2 shows a summarized block diagram of the extraction process as standardized by ETSI (ETSI 2003). Given this extraction process, MFCCs are subject to significant loss of information present in the original speech signal. The magnitude operation, for example, causes the loss of the phase information, while the Mel-filtering and the vector truncation (post-Discrete Cosine Transform) cause the loss of spectral detail (see, for example, Darch et al. 2005; Darch, Milner & Vaseghi 2006; Darch et al. 2007). In the framework of speech recognition, work such as Zheng & Zhang (2000) showed that speech recognition became much more robust once energy information was added to the MFCC vectors, and thus adding an energy coefficient through the logE operation, as seen below, became standard.

Figure 2

Block diagram of the ETSI Aurora standard for MFCC extraction.

Figure 3 below illustrates what the signal would look like if the truncated MFCC vectors were reversed back to a magnitude spectral representation. The blue line is the magnitude spectra derived from the MFCC, while the thinner line represents the original magnitude spectra.

Figure 3

Original (black line) and MFCC-derived (blue line) magnitude spectra.

Thus, MFCCs are a smoothed representation of the spectral envelope of the speech signal, and also contain one coefficient that keeps energy information. Still in the framework of speech recognition, it is noteworthy to mention that works by Darch and colleagues (Darch et al. 2005; Darch, Milner & Vaseghi 2006; Darch et al. 2007), have shown that formant frequencies can be accurately estimated (when compared formants calculated using LPC analysis) from MFCCs.

In Kaldi (and in ASR in general), feature vectors have hence 12 MFCC coefficients plus one energy feature. In addition to these 13 dimensions, to add some dynamics to the stationary observations, first order (the deltas) and second order (the deltas of deltas) derivatives of each MFCC and of the energy coefficient are calculated and added, resulting in a 39-dimensional feature vector.

The acoustic features extracted from the speech signal have to be modeled so that the ASR system can learn which feature vectors correspond to which phones. One of the most widely adopted modeling techniques, used in the present study, is the GMM-HMM acoustic modeling. The basic unit of this acoustic model is a context-independent (CI) phone, or a phone that is modelled independently from its neighboring phones. Alternatively, context-dependent (CD) phones²⁸ can also be used, but the latter fall outside of the scope of the present study (see comment in the Methodology section below). Each phone is usually modeled as a set of subphones or states, traditionally three of them (plus start and end states), so that in the word ‘cat’ [k æ t], for example, the phone [æ] would have three states (subphones), [æ]_1, [æ]_2, and [æ]_3. For each sub-phone there are two possible transitions, one to the next state, and one self-loop. Transitions from state i to state j are assigned probabilities, call them a_ij, or the transition probability. The word ‘cat’, would be then represented as in Figure 4.²⁹

Figure 4

Raw HMM for ‘cat’.

These states are generated using information from the lexicon (phonetic dictionary), and now need to be evaluated, in other words, we need to find out the likelihood of a sequence of observations given a specific HMM. The task is accomplished by associating a likelihood function to each state, modeled using a Mixture of Gaussians that generate probability density functions (PDFs). The continuous density HMM model for ‘cat’ would look as illustrated in Figure 5.

Figure 5

GMM-HMM for ‘cat’.

In this subsection, we briefly reviewed how Kaldi (Povey et al. 2011) specifically, and ASR tools in general, find the maximally probable sequence of words W in a language L to have generated the sequence of (acoustic) observations O made from the speech signal. Crucially, we reviewed the information extraction process used to generate MFCCs and work that showed that these vectors encode a smoothed representation of the spectral envelope, from which it is possible to accurately predict formant frequencies (e.g., Darch et al. 2005; Darch, Milner & Vaseghi 2006; Darch et al. 2007). Furthermore, MFCCs also encode an energy (logE) coefficient (ETSI 2003), which was shown to increase robustness in recognition.

4.1. Results for Word Tokens

Table 7 summarizes Accuracy and Error rates as a function of how the aligned word tokens compare to the expected word shape given the ground truth described in section 3 (Methodology), across the three experiments. From a classification perspective, it is expected that accuracy rates will decrease as the complexity of the experiment increases (where complexity is defined as the number of options the classifier can choose from in a given experiment), and the error rates will increase in the same direction, expectations which are both borne out from the data shown in the Table.

Analogously, the expectation that there would be only slight differences in performance⁴⁰ between the results for the training and the test data sets is confirmed by the data in Table 7. These results confirm that the model did not simply memorize the data used during the training pass, but rather learned to generalize over the data found in the training set, and it can thus be successfully used to evaluate new data. For each of the three experiments the accuracy rates (AR and CAR) are only slightly less optimistic for the test data sets, with differences in AR ranging from roughly 0.82 to 1.71 percentage points (for experiments 2ⁿ and n respectively). Conversely, the error rates (ER and CER), increase only very slightly in the test data sets (a difference that ranges from 0.18 to 0.38 percentage points).

The results presented in Table 7 are illustrated in Figures 6 and 7. Note that for all Figures within this section, the scale of the y-axes may change as best suited to provide a clearer visualization of the data.

The difference between Accuracy Rate (AR, Figure 6) and Composite Accuracy Rate (CAR, Figure 7) across the three experiments illustrates how the classifier expectedly changes predictions as it becomes more restricted with respect to where in a word token a vowel can be predicted to be stressed. In particular, the decrease in the difference between the AR and CAR from experiment 2ⁿ to experiment n+1 is noteworthy, because it indicates that for a sizable number of aligned tokens, the classifier predicted that both the vowel in citation position and some vowel in pretonic position were stressed in experiment 2ⁿ, and that when the choice was restricted to one vowel per word token in experiment n+1 (and also in experiment n), the classifier was significantly more likely to predict that the vowel in citation position was stressed (in lieu of predicting that a pretonic vowel was stressed).⁴¹

The visible decrease in the Error Rate (ER, or the rate of Mismatches) from experiment 2ⁿ to experiment n+1 reflects the proportion of aligned word tokens where the vowel in citation position and also a posttonic vowel were predicted to be stressed (roughly 2/3 of the total number of the total Mismatches in experiment 2ⁿ) and which were then mostly predicted to have a stressed vowel in citation position in experiment n+1.

The increase seen in the Composite Error Rate (CER) from experiment n to the other two experiments is a by-product of how the experiments were designed: in experiment n a word token where none of the vowels bear stress (the Unstressed option) is not a possibility. Given the amount of word tokens classified as Unstressed in experiments n+1 and 2ⁿ, it follows that a substantial number of those fell under the Matches and Partial Matches categories during experiment n, resulting in a visibly lower CER for this experiment.

In experiment 2ⁿ, for approximately 73% of the word tokens counted as Partial Matches, as mentioned above (a prediction of a stressed vowel in pretonic position) the vowel in citation position was also predicted to be stressed. This percentage, along with the decrease seen in the rate of Partial Matches from experiment 2ⁿ (see Figure 6) to experiment n+1 (roughly 9%) and to experiment n (approximately 7.4%), illustrates that ¾ of the times in which a vowel in pretonic position was predicted to be stressed the vowel in citation position was also predicted to be stressed, and that for the most part the prediction of stress locus fell back to the vowels in citation position when the choices for stress placement were restricted (experiments n and n+1).

A somewhat unexpected and interesting datum found in Table 7 and in Figure 6 is the fairly high rate of word tokens predicted to be completely Unstressed in experiments n+1 and 2ⁿ, totaling roughly 12.5% of the word tokens in the training data set, and about 13.2% of the word tokens in the test data set. Note that the rate remains virtually unchanged in both experiments, indicating that it is not a byproduct of the experiments’ design, or of chance classification, or of imbalances in the data sets, warranting a more detailed analysis.

The first thought that comes to mind is the possibility that the rate of Unstressed word tokens may be driven by (content) monosyllabic word tokens, which account for a sizeable portion of the corpus data, since for these there are only two choices, the (sole) vowel in the word token is either predicted to be stressed or unstressed. A glance at the distribution of word tokens predicted to be Unstressed in experiment 2ⁿ as a function of both the number of syllables and the position of stress as shown in Table 8 below, may prove informative.

The data in Table 8 shows that disyllabic and trisyllabic words account for approximately 2/3 of all the word tokens predicted to be Unstressed, and that penultimately stressed words make more than 60% of the tokens predicted to be Unstressed. In a complementary perspective, Figure 8 below shows the rate of word tokens predicted to be Unstressed with respect to the total number of word tokens of that size and stress locus.

Figure 8

Distribution of Unstressed tokens as a function of the number of syllables and stress position.

In the Figure, while roughly 14% of (content) monosyllables were predicted to be Unstressed, a little more than 16% of the disyllabic and almost 10% of the trisyllabic word tokens were also predicted to be so. Furthermore, there are also tetrasyllables (4.81% of the total number of tetrasyllabic words) and pentasyllables (3.6% of the total number of 5-syllable words) predicted to be Unstressed (and even a very small number of hexasyllabic words). Therefore, it seems that the rate of word tokens predicted to be Unstressed is not being driven uniquely by monosyllables, or, put differently, that the rate of word tokens predicted to be Unstressed is not a byproduct of the size of a word in syllables, either.

If the rate of word tokens predicted to be Unstressed is likely not a byproduct of the experiments’ design, nor is it particularly associated to the size of the word in syllables, then this datum must reflect a phenomenon that is linguistic in nature.⁴² A few possibilities arise: it may be the case that for these particular word tokens the spectral features and energy information used here alone do not capture stress information, and that perhaps another correlate (such as duration) is used to signal stress more unequivocally in these word tokens. It could also be that, for these particular word tokens, stress does not surface phonetically. These two hypotheses however, as phrased here, seem somewhat unlikely, since, first, the results so far have shown that spectral features and energy information capture stress fairly robustly, and second, it seems somewhat odd for stress to not surface phonetically in some random fashion. An alternative version for the former hypotheses would be that there is some phonological or prosodic process at work in these word tokens and that, as a consequence, either stress does not surface phonetically, or it is encoded using acoustic correlates that are not captured by the present model (in which case, the change in acoustic correlate in and of itself could potentially signal the phenomenon at hand). With respect to prosody, one contender that comes to mind is the position that the Unstressed word token occupies in the utterance. Figure 9 illustrates this distribution, showing the averages of Unstressed word tokens for each position of a given utterance. The labels closest to the X-axis represent the offset of the word within the utterance and the numbers below those represent the size of the utterance in number of words (if a position is missing for a given utterance size it means that all words in that position for all utterances of that size are monosyllabic function words). For each utterance size plotted in the graph (1-word to 10-word long), there are two or more distinct prompt sentences.

Figure 9

Distribution of Unstressed word tokens as a function of the position of the word in the utterance.

In the Figure above there is a trend whereby the rate of word tokens predicted to be Unstressed is visibly lower at the end of the utterance and is consistently higher towards mid-utterance position, which seems to indicate that word tokens are more likely to be predicted to be Unstressed in the prosodically weak positions of the utterance, and less likely so in positions of nuclear accent.⁴³ Note also that in the longer utterances (e.g., 8, 9, and 10 words), the pattern seems to repeat itself, in what could be the effects of two Intonational Phrases. Only a fully-fledged analysis of the trends seen in Figure 9 however,⁴⁴ would shed light on the true crux of the matter, which is, is stress in these Unstressed word tokens more phonological in nature (i.e., it does not surface phonetically), or does stress surface through different acoustic correlates, or combinations thereof, in the weaker positions of the utterance?

Since the results so far have shown that the outcomes are consistently more conservative (even if just slightly so) for the test data sets, the analyses presented in the remainder of this subsection will subsume these results only. Figure 10 presents the results for monosyllables in the corpus, summarized into two classes, that of function and of content words (results from experiment 2ⁿ). Although there is a large body of literature describing that monosyllabic function words, but not monosyllabic content words, are destressed cross-linguistically, there is no work in BP, to the best of this author’s knowledge, that shows this experimentally. The results fit these accounts well, as approximately 86% of the monosyllabic content word tokens were predicted to be stressed, but only around 35% of the function words fell in that prediction pattern. Note that these results are in agreement with phonological accounts whereby function words may be incorporated into adjacent phonological words.

Figure 10

Results (Test data set of experiment 2n) for function and content monosyllabic words.

Lastly, Figure 11 illustrates the breakdown of the results by stress locus, across all three experiments. Monosyllabic content words were not included in the plots, so to show all stress positions without the bias these words may introduce for ultimate stress.

Figure 11

Distribution of aligned word tokens as a function of stress position.

While the results for experiment n and n+1 show a trend whereby the Accuracy Rate increases the farther the stress locus falls from the right edge of the word, no clear pattern can be inferred form the results for experiment 2ⁿ. From this perspective it would be interesting to perform instrumental measurements of the acoustic correlates of the stressed vowels to see if a pattern indeed emerges with respect to how systematically these can tell apart the stressed vowels in the different positions of the word, in the lines of the word of Delgado Martins (1986, apud Magalhães, 2016) for EP, where it was found that duration and energy can be established instrumentally for ultimate and antepenultimate stress, but less systematically so for penultimate stress. Importantly, these results illustrate that the rate of word tokens predicted to have exactly the same shape as the citation word is very high across experimental conditions, for all three positions of stress.

4.2. The Classifier’s Discriminatory Skills

While the data discussed so far outlines how reliably stressed vowels are distinguished from the surrounding vowels in a given word token, they do not provide a detailed picture of discriminatory capacity of the model, or of how well it distinguishes between stressed and unstressed vowels given the ground truth. To do so, the data are summarized in confusion matrices and evaluated according to the metrics discussed in the Data Analysis subsection of this paper.

Table 9 shows that, analogously to the metrics discussed in the previous paragraphs, here too, for each of the three experiments, the rate of change between the metrics calculated for the training data sets and those calculated for the test data sets in each experiment is very small, not surpassing 2 percentage points for an individual metric. Figure 12 illustrates the results graphically.

Figure 12

Results for the classifier’s overall discriminatory skills.

The accuracy rates show that for all vowels predicted to be stressed and for all vowels predicted to be unstressed, the rate of truly stressed and truly unstressed vowels is very high across experiments and data sets (range 86%–94%). This means that the spectral features and energy information encoded in the MFCCs (modelled in HMM-GMMs) allow for the distinction between a stressed vowel and an unstressed vowel to be made with fairly high accuracy in BP.

The values for Sensitivity and Specificity indicate that, for all experiments and all data sets, the classifier is slightly better at correctly predicting that a vowel is unstressed when that vowel is expected to be unstressed, than it is at predicting that a vowel is stressed when that vowel is expected to be so. In an oversimplified way, these values indicate that, as represented by spectral and energy information encoded in MFCCs, unstressed vowels are slightly more likely to fit the model of an unstressed vowel than stressed vowels are likely to fit the model of a stressed vowel. Sensitivity is highest in experiment n and Specificity is highest in experiment n+1. Computationally this is a meaningful piece of information because it would help to decide which experiment is best suited if one is more interested in finding stressed or unstressed vowels in a speech corpus.

The results for Precision in the test data set show that when the classifier makes a positive prediction (a stressed vowel prediction), it is right 82% of the times in experiment 2ⁿ, 94% of the times in experiment n, and 96% of the times in experiment n+1. Precision is an important metric in as much as it illustrates the performance of the test: it could be, for example, that high values of Sensitivity were achieved because the classifier only makes positive predictions, and thus a considerable number of them are bound to be right.

The F1-scores provide an alternative measure of the classifier’s accuracy, a compromise between Precision and Sensitivity. The F1-scores⁴⁵ shown in Table 9 are almost as high as Accuracy for all three experiments, indicating that there is good balance between Precision and Sensitivity.

Recall from the Methodology section that the MCC (–1 ⩽ MCC ⩽ 1) measures the strength of correlation between the predictions and the ground truth taking into consideration the proportion of each class within the confusion matrix (TP, TN, FP, FN). The results achieved for the MCCs indicate a very strong correlation between predictions and ground truth across experiments and data sets. Put differently, the strength of correlation we see between predictions and the ground truth is not affected by the fact that language is imbalanced (the number of expected true negatives is much higher than the number of expected true positives).

Table 10 shows the Cohen’s Kappa Coefficients (Kappa), which represent the chance-corrected strength of the agreement between predictions and the ground truth. For each Kappa calculated, for the standard errors (SE) listed in the Table, there is a 95% chance that they would fall within the confidence intervals also shown in the Table. The Kappas for all experiments and all data sets are considered substantial (from 0.61 to 08) to almost perfect (0.81 to 1) in the interpretation given by Landis & Koch (1977) and are all considered excellent in the view of Cicchetti & Sparrow (1981).

The results for MCCs and Kappas for all experiments and all data sets help to ascertain that the strength of the agreement between predictions and the ground truth is neither a product of an imbalanced data set, nor the result of chance agreement.

In the following paragraphs, analogously to what was done in the previous subsection, a breakdown of the metrics calculated from the confusion matrices is detailed for the test data sets only. Figure 13 shows the metrics as a function of stress locus in a word. Monosyllabic content words were not included in the computations here to avoid bias on the metrics calculated for ultimate stress words (since False Positives (FP) and True Negatives (TN) will never occur in a monosyllabic word).

Figure 13

Discriminatory skills as a function of stress locus.

Recall from the results discussed in the previous subsection that there seemed to be a trend in experiments n and n+1 whereby AR showed that predictions became more accurate as the stress locus got farther from the right edge of the word, but that no pattern in this respect could be inferred from the results in experiment 2ⁿ. The graphs in the two first rows of Figure 13 show data that appear to corroborate this trend: across all experiments, all of the metrics shown in the graphs improve as the locus of stress moves away from the right edge of the word. In the lines of work like that of Gahl, Yao & Johnson (2012)—who showed that in (English) spontaneous speech words that are infrequent and have low phonological neighborhood density are more carefully articulated⁴⁶—one explanation for this asymmetry could be that penultimate and antepenultimate words in the WPC are more carefully articulated,⁴⁷ and as a consequence the vowels in them are better exemplars of both stressed and unstressed vowels, which results in more accurate predictions. An alternative explanation would be that spectral features and energy information capture stress more systematically as it falls farther from the right edge of the word, and instrumental measurements of other correlates of stress may or may not reproduce such asymmetry. Work for European Portuguese (EP) has shown an effect in these lines, in which duration and energy were found to be the most reliable correlates of stress in the language, but were instrumentally less systematic so for penultimate words (Delgado Martins 1986, apud Magalhães 2016) than for ultimate and antepenultimate words. It is worth noticing that, while the asymmetry between ultimate and penultimate words stems from a data set that contains 9,771 word tokens and 145 different word shapes in the former category, and 17,837 word tokens and 345 different word shapes of the latter category, in the case of antepenultimate words, only 365 word tokens and 11 word shapes determined the results here. Therefore, additional work using a data set that contains a larger number of antepenultimate word tokens and a greater variety of word shapes should shed additional light on this matter.

While Sensitivity and Specificity in Figure 13 mostly follow the general patterns seen in the overall results (Figure 12 earlier in the text), in experiment 2ⁿ, for antepenultimate words only, the rate of stressed vowels correctly identified as such among all vowels expected to be stressed becomes higher than the rate of unstressed vowels correctly identified as unstressed among all the truly unstressed vowels. This pattern likely stems from the design of the experiment, which allows vowels in all positions of a given word to be predicted as stressed. Indeed, in looking at the graphs in the second row for the same experiment, as Sensitivity increases for the antepenultimate words, Precision decreases, indicating that the ratio of correct positive predictions for all positive predictions made became lower. This trend is also visible on the F1-score for that category in the same experiment.

The F1-scores show that the balance between Precision and Sensitivity improves (F1-score plot is closer to Accuracy plot) for antepenultimate words in relation to penultimate and to ultimate words. The strength of the correlation between predictions and the ground truth, given by the MCCs, is also highest for antepenultimate words, as is the chance-corrected agreement between predictions and ground truth shown in yellow here.⁴⁸ The latter two metrics confirm, for all three stress loci, that the predictions are not a product of the imbalance in the class sizes, nor a result of chance classification.

On the second row of Figure 13, the graph on the right adds a plot for the Negative Predictive Value, the rate of correctly made predictions of unstressed vowels for all the unstressed vowel predictions made, which is a tradeoff with Specificity, a measure of the tests’ performances in finding relevant results for unstressed vowels. The results for NPV show that the rate of relevant negative predictions (an unstressed prediction) increases as citation stress locus falls farther from the right edge of the word in all experiments. NPV is highest for all stress loci in the least complex experiment (n) and is very close in value for the more complex experimental conditions (n+1 and 2ⁿ). The values for Precision show that there is a less clear trend in what concerns finding relevant positive results for the different stress loci, as the plot slightly dips for penultimate words before rising for antepenultimate words.

Because the various measures of accuracy presented here are very close in values, the differences are often harder to visualize, so the last graph in Figure 13 shows the False Positive Rate (FPR), the False Negative Rate (FNR) and the False Discovery Rate (FDR), a complementary view with respect to the accuracy metrics where changes in value are easier to spot. The values for FPR show that the rate of vowels incorrectly predicted to be stressed among all vowels expected to be unstressed is lower the farther the stress locus gets from the right edge of the word, for all experiments, meaning it is less likely that we find an incorrect prediction in antepenultimate then in penultimate and in ultimate words when the vowel is expected to be unstressed. The trend for FNR follows the same path of FPR, but in a much sharper fashion, where it is considerably less likely to find incorrect predictions among the vowels expected to be stressed in antepenultimate words.

Lastly, we move on to look at unstressed vowels separately in pre-tonic and in posttonic positions of the word. Since asymmetries between these positions have been previously reported in the literature (e.g., Câmara Jr. 1970; Major 1985) of Brazilian Portuguese, an analysis of the results obtained herein could potentially shed some additional light on the matter. Figure 14 summarizes the results for Specificity and for False Positive Rate for pre- and posttonic vowels. In the graph, recall that there are no posttonic positions in ultimate words and no pretonic positions in monosyllabic ultimate, disyllabic penultimate, and trisyllabic antepenultimate words.

Figure 14

Discriminatory skills for pretonic and posttonic vowels.

The results shown in Figure 14 present a fairly clear trend in which, for all experiments, and for all sizes of words within all stress loci, the rate of vowels correctly identified as unstressed among all the vowels expected to be unstressed (given by Specificity) is higher for posttonic positions. In other words, it is more likely for unstressed vowels in posttonic positions to be identified as such, than it is for unstressed vowels in pretonic positions of the word. The results from experiment 2ⁿ are the most telling, since in this experimental condition there are no restrictions to predicting that more than one vowel in the same word token is stressed. Indeed, the data indicate a clearer and sharper separation of pre- and posttonic vowels in this experiment. Showing the converse perspective, the graphs on the right side of the figure illustrate that the rate of False Positives (FPR) is higher for pretonic vowels in all experiments and word sizes within the three stress loci. The difference is again sharper in experiment 2ⁿ. Note that the asymmetry persists when there is imbalance between the number of pretonic and posttonic vowels in a word.

The question of whether the values of Specificity and Sensitivity in pretonic vowels can help to shed light on the nature and behavior of secondary stress in BP, while certainly a worthy one, falls beside the scope of the present study.⁴⁹

4.3. Discussion Synopsis

The results of the three experiments performed for the present study were discussed in this section. Results were averaged over the five folds of the experiments and then summarized and analyzed from two distinct and complementary perspectives, which provide insights on the syntagmatic and the paradigmatic nature of stressed and unstressed vowels as captured by a compressed representation of the speech signal that subsumes spectral features and energy information (MFCCs). The overall results for all metrics in both analyses, were, as anticipated, slightly less optimistic for the test data sets than for the training data sets, prompting the more detailed analyses to be summarized for the test data sets only.

From a syntagmatic perspective, the results have shown robustly that stressed vowels distinguish themselves from the surrounding vowels, even in a complex classification problem (experiment 2ⁿ), as shown by the Accuracy Rate (AR, or the rate of word tokens predicted to have the exact same shape of the citation form for that word) of 69.7% of the word tokens (19,498 in all). In the less complex experiments, AR approximates 93%. For experiments n and n+1, and only slightly so for experiment 2ⁿ, this syntagmatic effect is more noticeable the farther the stress locus gets from the right edge of the word—the rate of antepenultimate words that fits perfectly in the citation form of the word is larger than the rate of penultimate, than the rate of ultimate words.

Still from a syntagmatic perspective, the rate of tokens predicted to be completely Unstressed, a virtually steady 13% of all the word tokens in the corpus, in both experiment 2ⁿ and n+1, came as somewhat of a surprise. Having established that this rate is neither a byproduct of the design of the experiments, nor is it driven by monosyllabic words, a couple of options arise to explain these findings: one possibility is that, for a number of word tokens, stress cannot be captured using representations of the speech signal that subsume spectral features and energy information. Another possibility is that, in these tokens, stress is more of a phonological property, not necessarily surfacing phonetically. Independently of which of these two possibilities are correct, perhaps the crucial matter lies in trying to understand whether there is some phonological or prosodic process (or processes) at work in these Unstressed word tokens. In an attempt to do some inception work to answer this question, a brief analysis of Unstressed word tokens as a function of the position they occupy in the utterance was presented. This analysis showed a fairly clear trend in which Unstressed tokens are more likely to appear in the prosodically weaker positions of the utterance, for the different utterance sizes (in number of words) in the corpus. A much more detailed analysis is needed though, in trying to determine whether stress is more phonological in nature for these tokens, or whether, in these Unstressed tokens, stress is better captured through different acoustic correlates (in which case one might ask what the reason for such change would be: could it be to signal some other prosodic aspect of the utterance?).

A brief analysis of function and content monosyllabic words confirmed the trend widely reported in the literature cross-linguistically, which is that function monosyllabic words are generally destressed (only about 35% of them predicted to be stressed in experiment 2ⁿ), but not content monosyllabic words (approximately 86% of which were predicted to be stressed in experiment 2ⁿ). While these results are expected given the vast literature ascertaining so, there are no experimental studies looking at these two classes in BP, to the best of this author’s knowledge.

From a paradigmatic perspective the results proved to be yet more robust, with high values of Accuracy, Sensitivity and Specificity in all three experiments for all stress loci. These results revealed that stressed and unstressed vowels, as captured by MFCC representations of the signal and modelled in the HMM-GMM framework, are systematically different from one another. The values for the Matthew’s Correlation Coefficients show that the correlation between predictions and the ground truth is still very strong after taking the imbalances in the data set into account, and the Cohen’s Kappa Coefficients indicated a very strong to almost perfect chance-corrected correlation between predictions and ground truth.

Noteworthy, also from a paradigmatic perspective, is the fact that there is a clearer trend in which stressed vowels and unstressed vowels appear to be more distinctly so as the locus of stress in the citation form of the word moves away from its right edge. Differently put, both stressed and unstressed vowels are more likely to be accurately predicted as such in antepenultimate words then they are in penultimate words, then they are in ultimate words. A parallel asymmetry was shown to exist for penultimate words in European Portuguese in the work of Delgado Martins (1986, apud Magalhães 2016). Whether or not these asymmetries are rooted in cognitive processes, in line with work such as Gahl, Yao & Johnson (2012), and whether they hold after studies on larger data sets establish instrumentally the acoustic correlates of stress in point, are some of the questions that remain open.

Lastly, an analysis of Specificity and of False Positive Rate (FPR) for pre- and posttonic vowels separately revealed an asymmetry between these positions in the word, as previously reported in the literature for BP (e.g., Major 1985; Câmara Jr. 1970). For all experiments and all word sizes within all stress loci, posttonic vowels are more likely to be predicted to be unstressed than pretonic vowels are so. The effect holds despite the imbalance in the number of pre- and posttonic vowels in words of different sizes and stress locus.

Several metrics in both analyses—including the rate of Unstressed word tokens, the change in rates of Partial Matches and Error, Accuracy rates for monosyllabic content word tokens, the progression of Precision, Sensitivity and Specificity, among others—indicated inter-experiment agreement where expected. Conversely, other metrics confirmed differences where they would be expected, given the design of each experiment—e.g., a perfect agreement for monosyllabic content words in experiment n, lower word token Accuracy Rate (perfect matches) in experiment 2ⁿ, and Accuracy (from the confusion matrix) of 1 for monosyllabic content words in experiment n, among others.